Common transformations
🎯You'll be able to
Apply filter, map, group-and-total, look-up and de-duplicate patterns to real workflow data.
🏠The everyday picture
A shopkeeper doing the evening accounts repeats the same few moves: add up the day's sales, group them by product, cross out duplicate bills, and look up prices in the rate list. Most code steps repeat the same few moves too.
📘What it really is
- Five patterns cover most code steps:
- Filter: keep items that meet a rule (
orders.filter(o => o.paid)). - Map: change each item (
orders.map(o => ({ ...o, rupees: o.amount_paise / 100 }))). - Group and total: collect items by a key, such as customer or product.
- Deduplicate: keep one item per key, such as phone number or order ID.
-
Look up: match a value against a table, such as SKU to product name.
-
Decide which record wins when deduplicating: the newest, the most complete, or the first. Write the rule down.
-
Keep lookup tables in one place, such as a sheet or a small object at the top of the code, so they can be updated without touching the logic.
-
Handle missing values on purpose: a missing price should stop or flag the item, not become zero.
📝A worked example
Weekly CRM tidy-up for Sharma Textiles, one of the three path projects. Input: 412 contacts with E.164 phones from DA4.
const contacts = $input.all().map(i => i.json);
const byPhone = {};
for (const c of contacts) {
const key = c.phone_e164;
if (!key) continue; // invalid numbers went to review in DA4
const current = byPhone[key];
if (!current || c.updated_at > current.updated_at) byPhone[key] = c; // newest wins
}
const kept = Object.values(byPhone);
return [
...kept.map(c => ({ json: { ...c, action: 'keep' } })),
{ json: { action: 'report', before: contacts.length, after: kept.length } }
];
The run reports 412 contacts before and 371 after: 41 duplicates to merge. Because the rule is "newest wins", the report also lists which record was kept for each phone number, so staff can check.
🌍Where it's used
- CRM hygiene: merging duplicate contacts.
- Stock and orders: totals per product or per day.
- Pricing: looking up rates and discounts.
- Lead lists: filtering by region or score before a campaign.
📝Case scenario
After a deduplication step, a long-standing customer's GST number disappears from the CRM. The step kept the newest record, which came from a quick enquiry form without a GST field. You change the rule to merge fields: keep the newest value for each field, but don't overwrite a filled field with an empty one. You rerun on a copy first and compare.
🖱️Try it
Using your 20 cleaned contacts from DA4, write a Code node that removes duplicates by phone number using "newest wins", and returns a report item with the before and after counts. Then add the rule "don't overwrite a filled field with an empty one".
⚠️Common mistakes
- De-duplicating on raw values. Sign: "98765 43210" and "+919876543210" both survive. Fix: normalise before comparing.
- Grouping without checking totals. Sign: a group's sum doesn't match the source. Fix: compare totals before and after.
🛠️For your project
Project 3's duplicate finding is group-and-merge on cleaned phones and lower-case emails. Write and test it on data/contacts.json.
✅Quick check
Name the five common transformation patterns.
Show the answer
Filter, map, group and total, deduplicate, and look up.
Why should a missing price stop or flag an item rather than become zero?
Show the answer
A zero price would quietly produce wrong totals or free orders.