bastianplsfix

← JavaScript + TypeScript

Transforming arrays

Four methods cover almost everything you do to the contents of an array. map for one output element per input element. filter for some of the input elements. flatMap for zero or many output elements per input element. reduce for one value out of all of them.

None of them changes the array they are called on. Each returns something new, which is why they chain, and which is the reason to prefer them to a loop that mutates as it goes; the mutating arrays page covers the operations that do change an array. They all take the same callback, and it receives three arguments, not one. That fact is worth learning early, because it causes the first bug in this entry.

Create programs/transforming-arrays.test.ts for this reference and keep it open. Start it with the assertion helpers used on this page:

import {
  assert,
  assertEquals,
  assertFalse,
  assertStrictEquals,
  assertThrows,
} from "@std/assert";

Follow the page as you add and revise the runnable examples below that import.

the four, at a glance

Deno.test("the four, at a glance", () => {
  assertEquals([1, 2, 3].map((n) => n * 3), [3, 6, 9]);
  assertEquals([-1, 2, -7, 6].filter((n) => n >= 0), [2, 6]);
  assertEquals(["ab", "c"].flatMap((s) => [...s]), ["a", "b", "c"]);
});
Check programs/transforming-arrays.test.ts
running 1 test from ./programs/transforming-arrays.test.ts
the four, at a glance ... ok (296µs)

ok | 1 passed | 0 failed (1ms)

map applies the callback to each element and collects the results, so the output is always the same length as the input. filter keeps the elements for which the callback returns something truthy, by the rules of the truthiness page. flatMap expects an array back from the callback and concatenates them all, so each input can produce any number of outputs, including none. reduce gets its own steps further down.

the callback signature

Deno.test("the callback signature", () => {
  const described = ["a", "b"].map(
    (value, index, whole) => `${value} at ${index} of ${whole.length}`,
  );

  assertEquals(described, ["a at 0 of 2", "b at 1 of 2"]);
});
the callback signature ... ok (38µs)

Every one of these methods passes the element, its index, and the whole array. You are free to ignore the last two, and almost always do. The shape to hold in mind is (value: T, index: number, array: T[]) => R, and two steps from now the second parameter causes a real bug.

searching, and answering

Deno.test("searching, and answering", () => {
  const amounts = [-1, 2, -3];

  assertStrictEquals(amounts.find((n) => n < 0), -1);
  assertStrictEquals(amounts.findLast((n) => n < 0), -3);
  assertStrictEquals(amounts.findIndex((n) => n < 0), 0);
  assertStrictEquals(amounts.findLastIndex((n) => n < 0), 2);

  assertStrictEquals(amounts.find((n) => n > 100), undefined);
  assertStrictEquals(amounts.findIndex((n) => n > 100), -1);

  assert(amounts.some((n) => n < 0));
  assertFalse(amounts.every((n) => n < 0));
});
searching, and answering ... ok (76µs)

Four ways to find an element and two ways to ask a question about all of them. Note the middle pair, where nothing matches: find reports failure as undefined and findIndex reports it as -1, which is a sentinel with all the problems that implies, and the sentinels page is about exactly this. forEach also exists and takes the same callback; prefer for-of, for the reasons the loops page gives in .forEach() cannot stop.

the index argument is why map(parseInt) fails

Number parses by name, so predict what parseInt does in the same position:

Deno.test("the index argument is why map(parseInt) fails", () => {
  assertEquals(["1", "2", "3"].map(Number), [1, 2, 3]);
  assertEquals(["1", "2", "3"].map(parseInt), [1, 2, 3]);
});
Check programs/transforming-arrays.test.ts
running 4 tests from ./programs/transforming-arrays.test.ts
...
the index argument is why map(parseInt) fails ... FAILED (9ms)

 ERRORS

the index argument is why map(parseInt) fails => ./programs/transforming-arrays.test.ts:38:6
error: AssertionError: Values are not equal.

    [Diff] Actual / Expected

    [
      1,
-     NaN,
-     NaN,
+     2,
+     3,
    ]

 FAILURES

the index argument is why map(parseInt) fails => ./programs/transforming-arrays.test.ts:38:6

FAILED | 3 passed | 1 failed (9ms)

error: Test failed

Two thirds of a reasonable-looking line is NaN. Number takes one argument, so passing it by name works. parseInt takes two, the second being the radix, and map supplies the index as a second argument to everything it calls, so element one is parsed in base 1 and element two in base 2. Correct the prediction and pin the calls that actually happened:

Deno.test("the index argument is why map(parseInt) fails", () => {
  assertEquals(["1", "2", "3"].map(Number), [1, 2, 3]);
  assertEquals(["1", "2", "3"].map(parseInt), [1, NaN, NaN]);

  assertStrictEquals(parseInt("1", 0), 1);
  assert(Number.isNaN(parseInt("2", 1)));
  assert(Number.isNaN(parseInt("3", 2)));

  assertEquals(["1", "2", "3"].map((text) => parseInt(text, 10)), [1, 2, 3]);
});
the index argument is why map(parseInt) fails ... ok (72µs)

A radix of 0 means "guess", there is no base 1, and 3 is not a binary digit. TypeScript does not object, and it is right not to: the extra parameters are genuinely there, and a function that accepts a number where a number is passed is being called correctly. The mistake is at a level the checker does not model. The lesson generalises past parseInt: passing a function by name to a callback position hands it every argument the method supplies, so when you did not write the function, wrap it, as the last line does. .map((text) => parseInt(text, 10)) says what you meant and cannot be surprised.

filter narrows the element type

Deno.test("filter narrows the element type", () => {
  const mixed: (string | undefined)[] = ["a", undefined, "b"];

  const defined = mixed.filter((value) => value !== undefined);
  assertEquals(defined.map((value) => value.toUpperCase()), ["A", "B"]);

  function isString(value: unknown): value is string {
    return typeof value === "string";
  }

  const values: unknown[] = ["a", 1, "b"];
  assertEquals(values.filter(isString).map((s) => s.toUpperCase()), ["A", "B"]);
});
filter narrows the element type ... ok (43µs)

mixed is (string | undefined)[], and defined is string[]: TypeScript reads the callback body, sees that it is a type test, and infers a type predicate from it, so filtering out the empty case narrows the array, which is what you always wanted it to do. The .toUpperCase() needing no check and no cast is the proof. This inference is recent, and you will meet code written before it, where the same line needed the hand-written form below: value is string in the return position is a type predicate, the tool from the unions and narrowing page's a predicate teaches the checker a check of your own, and narrowing unknown[] still needs it, as does any filter whose condition is more involved than a comparison.

flatMap filters and maps in one pass

Deno.test("flatMap filters and maps in one pass", () => {
  const raw = ["1", "not a number", "3"];

  const parsed = raw.flatMap((text) => {
    const value = Number(text);
    return Number.isNaN(value) ? [] : [value];
  });

  assertEquals(parsed, [1, 3]);

  assertEquals(["a", "b"].flatMap((s) => s === "a" ? [s, s] : s), [
    "a",
    "a",
    "b",
  ]);
  assertEquals([1, [2, [3]]].flat(), [1, 2, [3]]);
  assertEquals([1, [2, [3]]].flat(2), [1, 2, 3]);
});
flatMap filters and maps in one pass ... ok (51µs)

Return a one-element array to keep the element, an empty array to drop it. One pass, one callback, and the parsing happens exactly once, where a filter followed by a map would either parse twice or need a separate step to carry the parsed value along. Think of flatMap as zero or many rather than as flattening: the flattening framing suggests its use is nested data, and the useful case is this one. A callback may also return a plain value rather than an array, which is occasionally convenient and mostly a way to write a confusing line. flat is the flattening half on its own, with a depth that defaults to one, and its type is honest about that: [1, [2, [3]]].flat() is (number | number[])[], because one level down there are still arrays. Flattening an unknown number of levels is not something the type system can follow, which is a fair warning about data shaped that way.

reduce, one call at a time

Deno.test("reduce, one call at a time", () => {
  const calls: string[] = [];

  const total = [1, 2, 3].reduce((sum, n) => {
    calls.push(`(${sum}, ${n}) -> ${sum + n}`);
    return sum + n;
  }, 0);

  assertStrictEquals(total, 6);
  assertEquals(calls, ["(0, 1) -> 1", "(1, 2) -> 3", "(3, 3) -> 6"]);
});
reduce, one call at a time ... ok (29µs)

The log is the whole explanation. reduce calls your callback once per element with two things: the accumulator, which is whatever the previous call returned, and the current element. The first call gets the initial value you passed, and the last call's result is the result of reduce. Everything else about reduce follows from that, including that the accumulator need not be the same type as the elements. It is a fold: many values in, one value out, and the one value can be a number, a string, an object, or another array.

omitting the initial value is a different function

Three calls that work, then predict the fourth:

Deno.test("omitting the initial value is a different function", () => {
  const add = (a: number, b: number) => a + b;

  assertStrictEquals([1, 2].reduce(add, 100), 103);
  assertStrictEquals([1, 2].reduce(add), 3);
  assertStrictEquals(([] as number[]).reduce(add, 100), 100);

  assertStrictEquals(([] as number[]).reduce(add), 0);
});
Check programs/transforming-arrays.test.ts
running 8 tests from ./programs/transforming-arrays.test.ts
...
omitting the initial value is a different function ... FAILED (274µs)

 ERRORS

omitting the initial value is a different function => ./programs/transforming-arrays.test.ts:94:6
error: TypeError: Reduce of empty array with no initial value
  assertStrictEquals(([] as number[]).reduce(add), 0);
                                      ^

 FAILURES

omitting the initial value is a different function => ./programs/transforming-arrays.test.ts:94:6

FAILED | 7 passed | 1 failed (2ms)

error: Test failed

It throws. With an initial value, an empty array gives you that value back and the callback is never called. Without one, the first element becomes the accumulator and iteration starts at the second, a reasonable shortcut for a sum, and on an empty array there is nothing to return, so the language throws. That is the right decision by the language and a bad surprise in production, because it fires only for the input you did not test with. Pin the crash:

Deno.test("omitting the initial value is a different function", () => {
  const add = (a: number, b: number) => a + b;

  assertStrictEquals([1, 2].reduce(add, 100), 103);
  assertStrictEquals([1, 2].reduce(add), 3);
  assertStrictEquals(([] as number[]).reduce(add, 100), 100);

  assertThrows(
    () => ([] as number[]).reduce(add),
    TypeError,
    "Reduce of empty array with no initial value",
  );
});
omitting the initial value is a different function ... ok (296µs)

Always pass the initial value. It costs three characters, it documents the type of the accumulator, and it removes a whole class of failure.

an accumulator of another shape needs a type argument

Reduce numbers into an array, spelled the obvious way:

Deno.test("an accumulator of another shape needs a type argument", () => {
  const values = [1, 2, 3];

  const doubled = values.reduce((acc, n) => [...acc, n * 2], []);

  assertEquals(doubled, [2, 4, 6]);
});
Check programs/transforming-arrays.test.ts
TS2769 [ERROR]: No overload matches this call.
  Overload 1 of 3, '(callbackfn: (previousValue: number, currentValue: number, currentIndex: number, array: number[]) => number, initialValue: number): number', gave the following error.
    Type 'number[]' is not assignable to type 'number'.
  const doubled = values.reduce((acc, n) => [...acc, n * 2], []);
                                            ~~~~~~~~~~~~~~~
    at file:///programs/transforming-arrays.test.ts:111:45

    The expected type comes from the return type of this signature.
        reduce(callbackfn: (previousValue: T, currentValue: T, currentIndex: number, array: T[]) => T, initialValue: T): T;
                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        at asset:///lib.es5.d.ts:1487:24

TS2345 [ERROR]: Argument of type 'number[]' is not assignable to parameter of type 'number'.
  assertEquals(doubled, [2, 4, 6]);
                        ~~~~~~~~~
    at file:///programs/transforming-arrays.test.ts:113:25

Found 2 errors.

error: Type checking failed.

This is the reduce error everyone meets, with a second error trailing it because the failed inference poisoned doubled too. Read the quoted overload and it explains itself: with an array of numbers, the first overload assumes the accumulator is a number too, so an initial value of [] gets inferred as number[] where number was expected, and the callback returning an array is reported as the problem. The accumulator is allowed to be a different type; the checker just cannot guess that from []. Tell it with a type argument:

Deno.test("an accumulator of another shape needs a type argument", () => {
  const values = [1, 2, 3];

  assertEquals(values.reduce<number[]>((acc, n) => [...acc, n * 2], []), [
    2,
    4,
    6,
  ]);

  const index = ["apple", "avocado", "beet"].reduce<Record<string, string[]>>(
    (acc, word) => {
      (acc[word[0]] ??= []).push(word);
      return acc;
    },
    {},
  );

  assertEquals(index, { a: ["apple", "avocado"], b: ["beet"] });
});
an accumulator of another shape needs a type argument ... ok (41µs)

The type argument is the better habit over annotating the parameter, because it says what the whole call produces rather than what one parameter is, and it reads well when the accumulator is the point, as in index. That is reduce doing the job it is best at; for grouping specifically, reach for Object.groupBy or Map.groupBy, which the objects as dictionaries page covers.

a reduce cannot stop, so it carries the answer

Deno.test("a reduce cannot stop, so it carries the answer", () => {
  const NOT_FOUND = -1;

  function indexOfViaReduce(values: string[], wanted: string): number {
    return values.reduce((found, value, index) => {
      if (found !== NOT_FOUND) return found;
      return value === wanted ? index : NOT_FOUND;
    }, NOT_FOUND);
  }

  function indexOfViaLoop(values: string[], wanted: string): number {
    for (const [index, value] of values.entries()) {
      if (value === wanted) return index;
    }
    return NOT_FOUND;
  }

  assertStrictEquals(indexOfViaReduce(["a", "b"], "b"), 1);
  assertStrictEquals(indexOfViaLoop(["a", "b"], "b"), 1);
  assertStrictEquals(indexOfViaReduce(["a"], "z"), NOT_FOUND);

  let checked = 0;
  [1, -2, 3].every((n) => {
    checked++;
    return n > 0;
  });
  assertStrictEquals(checked, 2);

  const empty: number[] = [];
  assert(empty.every((n) => n > 0));
  assertFalse(empty.some((n) => n > 0));
});
a reduce cannot stop, so it carries the answer ... ok (72µs)

Two implementations of indexOf, and the difference is not style. The reduce version visits every element after it has the answer, and it needs its first line to stop the answer being overwritten by later elements: the guard exists only because the loop cannot end, while the loop version returns. That guard clause is the signal. When a reduce grows one, you are folding something that is not a fold, and a for-of will be shorter and faster and easier to read. reduce is right when every element genuinely contributes.

The searching methods do stop, and so do every and some: checked stayed at 2 because every stops at the first falsy result, and some stops at the first truthy one. Then note the empty array: every is true and some is false, which is the standard reading of "for all" and "there exists" over nothing, and also a real source of bugs, because every(isValid) returning true reads as reassurance and here means only that there was nothing to validate. If empty is a case you care about, check the length first.

includes and indexOf disagree twice

Deno.test("includes and indexOf disagree twice", () => {
  assert([NaN].includes(NaN));
  assertStrictEquals([NaN].indexOf(NaN), -1);

  const holed = ["a", , "b"];
  assert(holed.includes(undefined));
  assertStrictEquals(holed.indexOf(undefined), -1);
});
includes and indexOf disagree twice ... ok (22µs)

Both search for a value rather than running a callback, and they use different comparisons. indexOf uses ===, which says NaN is not NaN; includes uses SameValueZero, which says it is, the same family of comparison rules the equality page lays out. The second pair is the hole rule from the arrays page showing up here: includes treats a hole as undefined and finds it, indexOf skips holes and does not. So includes is the one you want when asking whether a value is present, and indexOf is for when you need the position.

asking whether a string is one of a known set

A reasonable thing to want:

Deno.test("asking whether a string is one of a known set", () => {
  const levels = ["debug", "info"] as const;

  function isKnown(input: string): boolean {
    return levels.includes(input);
  }

  assert(isKnown("info"));
});
Check programs/transforming-arrays.test.ts
TS2345 [ERROR]: Argument of type 'string' is not assignable to parameter of type '"debug" | "info"'.
    return levels.includes(input);
                           ~~~~~
    at file:///programs/transforming-arrays.test.ts:174:28

error: Type checking failed.

A compile error, because as const gives levels the type readonly ["debug", "info"], so its includes accepts only "debug" | "info", and the whole point of the call was to find out whether input is one of those. The reasoning is not wrong, just aimed at a different mistake: includes takes the element type so that searching an array of numbers for a string can be caught, and that is usually what you want. The fix is to widen the array rather than cast the value, and to get a narrowing out of it while you are there:

Deno.test("asking whether a string is one of a known set", () => {
  const levels = ["debug", "info"] as const;
  type Level = typeof levels[number];

  function isLevel(input: string): input is Level {
    return (levels as readonly string[]).includes(input);
  }

  assert(isLevel("info"));
  assertFalse(isLevel("shout"));

  const raw = "debug";
  if (isLevel(raw)) {
    const level: Level = raw;
    assertStrictEquals(level, "debug");
  }
});
asking whether a string is one of a known set ... ok (22µs)

typeof levels[number] is the union of the element types, which is how you get "debug" | "info" from the array without writing it twice. The cast to readonly string[] is confined to one line inside a function whose signature says what it is for, and every caller gets narrowing for free.

chaining materialises each step

Deno.test("chaining materialises each step", () => {
  const steps: string[] = [];

  const result = [1, 2, 3, 4]
    .filter((n) => {
      steps.push(`filter ${n}`);
      return n % 2 === 0;
    })
    .map((n) => {
      steps.push(`map ${n}`);
      return n * 10;
    });

  assertEquals(result, [20, 40]);
  assertEquals(steps, [
    "filter 1",
    "filter 2",
    "filter 3",
    "filter 4",
    "map 2",
    "map 4",
  ]);

  const live = [1, 2, 3];
  const mapped = live.map((n, index, whole) => {
    if (index === 0) whole[2] = 99;
    return n;
  });
  assertEquals(mapped, [1, 2, 99]);
});
chaining materialises each step ... ok (42µs)

Read the log: all the filtering happens, then all the mapping. The intermediate array of two elements really exists, and a longer chain builds one array per step. For four elements that is free; for a large collection it is work and memory you did not need, and for an endless one it never finishes. The iterator helpers page is the alternative and argues this from the other side in nothing happens until something pulls: the same chain written on an iterator pulls one element through every step and materialises nothing. The recommendation from this side is unchanged: arrays are right until the collection is large, endless, or expensive per element.

The last third is one closing fact about the third argument: it is the array itself, not a copy, so a callback can change what later calls will see, which is why mapped ends in 99. Worth knowing so you can recognise it, and worth never doing. If the transformation needs the whole array, read from it and let map build the result.

In practice