bastianplsfix

Iterables and iterators

There is one protocol, and everything that walks over values consumes it. A value is iterable if it has a method under the key Symbol.iterator that returns an iterator, and an iterator is an object with a next() method handing back { value, done }. That is the whole arrangement, and the key it hangs on already appeared on the symbols page.

for-of, spreading into an array, array destructuring, Array.from, new Set(...), new Map(...), Promise.all, and yield* all ask that one question. Arrays are not privileged here; they implement the protocol, and so can anything you write.

Two consequences, and they are the reason to know this at all. Anything you write becomes loopable, spreadable, and destructurable by adding one method. And anything that arrives as an iterable may be single-use, or endless, and its type will not tell you which.

Create programs/iterables-and-iterators.test.ts for this reference and keep it open. Start it with the assertion helpers used on this page:

import { assertEquals, assertStrictEquals, assertThrows } from "@std/assert";

Follow the page as you add and revise the runnable examples below that import.

the iterable makes a walker that holds the position

The protocol has two halves, so the first step builds both by hand:

Deno.test("the iterable makes a walker that holds the position", () => {
const countdown: Iterable<number> = {
[Symbol.iterator](): Iterator<number> {
let n = 3;
return {
next(): IteratorResult<number> {
return n > 0
? { value: n--, done: false }
: { value: undefined, done: true };
},
};
},
};

assertEquals([...countdown], [3, 2, 1]);
assertEquals([...countdown], [3, 2, 1]);
});
Check programs/iterables-and-iterators.test.ts
running 1 test from ./programs/iterables-and-iterators.test.ts
the iterable makes a walker that holds the position ... ok (0ms)

ok | 1 passed | 0 failed (1ms)

The iterable is the outer object, and its one job is to produce a walker. The iterator is the walker, and it holds the position, which here is the variable n closed over by next(). Follow one spread through:

  1. [...countdown] asks countdown for an iterator by calling [Symbol.iterator](), which creates a fresh n holding 3 and returns the object whose next() can see it.
  2. Each next() call finds n > 0 true, hands out { value: n, done: false }, and moves n down by one, so the spread collects 3, then 2, then 1.
  3. When n reaches 0, next() returns { value: undefined, done: true }, and done: true is the signal that makes the spread stop asking.

Keeping the two halves apart is what makes the second spread start over: the second [...countdown] calls [Symbol.iterator]() again, and each call builds a new n. The result object's shape is the one the sentinels page derives from first principles, and the argument for why the end of input is a wrapper rather than a special value lives there, in iteration already made this decision.

a generator writes the walker for you

Deno.test("a generator writes the walker for you", () => {
function* counted(): Generator<number> {
for (let n = 3; n > 0; n--) yield n;
}

assertEquals([...counted()], [3, 2, 1]);
});
a generator writes the walker for you ... ok (0ms)

The same values, with none of the bookkeeping. function* declares a generator, calling it returns an object that is both an iterator and an iterable, and yield hands out one value while keeping the position for you, exactly the job n and the hand-written next() did above. This is the argument for generators existing: writing an iterator by hand is mechanical work that a language feature can do.

There is one difference between countdown and counted(), it is not cosmetic, and it gets its own step in one-shot and reusable are both spelled Iterable.

the standard library is already on the protocol

Deno.test("the standard library is already on the protocol", () => {
assertEquals([...new Set(["x", "x", "y"])], ["x", "y"]);
assertEquals([...new Map([["a", 1]])], [["a", 1]]);
assertEquals(
[...new URLSearchParams("a=1&b=2")],
[["a", "1"], ["b", "2"]],
);
assertEquals([..."a๐Ÿ™‚"], ["a", "๐Ÿ™‚"]);
assertEquals([...[1, 2].entries()], [[0, 1], [1, 2]]);
});
the standard library is already on the protocol ... ok (0ms)

A Set yields its members, with the duplicate already gone because a Set never held it twice. A Map and a URLSearchParams yield [key, value] pairs. A string yields code points rather than the UTF-16 units .length counts, the split the text and characters page measured in iteration gives you code points, and the loops page walked with for-of in strings, Maps, and Sets implement the same protocol. The .entries(), .keys(), and .values() family return iterators, and those are iterable too, for a reason two steps down.

every consumer asks the same question

Deno.test("every consumer asks the same question", () => {
const countdown: Iterable<number> = {
[Symbol.iterator](): Iterator<number> {
let n = 3;
return {
next(): IteratorResult<number> {
return n > 0
? { value: n--, done: false }
: { value: undefined, done: true };
},
};
},
};

const seen: number[] = [];
for (const value of countdown) seen.push(value);
assertEquals(seen, [3, 2, 1]);

const [head, next] = countdown;
assertEquals([head, next], [3, 2]);

assertEquals(Array.from(countdown), [3, 2, 1]);
assertEquals([...new Set(countdown)], [3, 2, 1]);
});
every consumer asks the same question ... ok (1ms)

One hand-written object, four unrelated consumers, no adapters. The for-of is the loop from the loops page, now aimed at an object we built ourselves. The const [head, next] = countdown line is array destructuring, the pattern the destructuring page takes apart, and it consumes the protocol too: it asked for a walker, took two values, and stopped. Array.from builds an array, and new Set builds a set, each by asking the same one question. That is the whole payoff of a protocol over a base class: none of these consumers know anything about countdown except that it answers [Symbol.iterator]().

an iterator is also an iterable, deliberately

Deno.test("an iterator is also an iterable, deliberately", () => {
const walker = ["a", "b", "c"][Symbol.iterator]();

assertStrictEquals(walker.next().value, "a");
assertEquals([...walker], ["b", "c"]);
assertStrictEquals(walker[Symbol.iterator](), walker);
});
an iterator is also an iterable, deliberately ... ok (0ms)

The spread over walker produced ["b", "c"], not ["a", "b", "c"], because the walker had already handed out "a" to the next() call, and spreading it picked up where it stopped rather than starting over. That works because every built-in iterator carries a [Symbol.iterator]() that returns itself, which is what the last assertion pins: asking the walker for a walker hands back the very same object. This convention is what lets a half-consumed iterator be handed to for-of or a spread, and it is why .entries() could be written straight into a loop head on the loops page in .entries() keeps the index inside for-of. TypeScript names the combination IterableIterator<T>.

It also means iterable and iterator are not exclusive categories, which is most of why the two words get muddled in conversation. The distinction that actually matters is not what an object is called. It is whether asking it for an iterator gives you a fresh one, and the next step measures what happens when it does not.

one-shot and reusable are both spelled Iterable

A generator and an array, spread twice each. Predict all four results:

Deno.test("one-shot and reusable are both spelled Iterable", () => {
function* counted(): Generator<number> {
for (let n = 3; n > 0; n--) yield n;
}

const generated = counted();
assertEquals([...generated], [3, 2, 1]);
assertEquals([...generated], [3, 2, 1]);

const array = [3, 2, 1];
assertEquals([...array], [3, 2, 1]);
assertEquals([...array], [3, 2, 1]);

function twice(values: Iterable<string>): string[] {
const held = [...values];
return [...held, ...held];
}
assertEquals(twice(["a", "b"]), ["a", "b", "a", "b"]);
});
Check programs/iterables-and-iterators.test.ts
running 6 tests from ./programs/iterables-and-iterators.test.ts
...
one-shot and reusable are both spelled Iterable ... FAILED (8ms)

ERRORS

one-shot and reusable are both spelled Iterable => ./programs/iterables-and-iterators.test.ts:74:11
error: AssertionError: Values are not equal.

[Diff] Actual / Expected

+ [
+ 3,
+ 2,
+ 1,
+ ]
- []

FAILURES

one-shot and reusable are both spelled Iterable => ./programs/iterables-and-iterators.test.ts:74:11

FAILED | 5 passed | 1 failed (11ms)

error: Test failed

The second spread of generated produced [], and nothing complained. This is the trap, because both generated and array are Iterable<number>, and the type says nothing about which one you are holding. Hand a generator to two functions that each loop over it and the second one sees an empty sequence, with no error raised anywhere.

The mechanism is the one the first two steps built:

  1. An array's [Symbol.iterator]() builds a new walker every time, so the second spread of array gets a fresh position starting at the front.
  2. A generator is the walker, and the previous step showed what a walker's [Symbol.iterator]() returns: itself. There is nothing to build, so there is nothing to reset.
  3. The second spread of generated therefore asks a finished walker for more, next() answers { done: true } immediately, and the spread collects nothing.

Reusable means the iterable and the iterator are different objects. Correct the prediction to []:

one-shot and reusable are both spelled Iterable ... ok (0ms)

The twice function is the discipline this forces: a function that needs two passes pays for the values once, at the top, deliberately, rather than discovering the problem from a second loop that quietly did nothing. This matters more than it used to, because chaining iterator helpers is now idiomatic, the iterator helpers page covers them, and every chain has a single-use iterator at the bottom of it.

leaving early closes the iterator

Deno.test("leaving early closes the iterator", () => {
const events: string[] = [];
function* withCleanup(): Generator<number> {
try {
yield 1;
yield 2;
} finally {
events.push("closed");
}
}

for (const n of withCleanup()) {
events.push(`saw ${n}`);
break;
}
assertEquals(events, ["saw 1", "closed"]);

function firstOnly(): number {
for (const n of withCleanup()) return n;
return -1;
}
assertStrictEquals(firstOnly(), 1);
assertEquals(events, ["saw 1", "closed", "closed"]);
});
leaving early closes the iterator ... ok (0ms)

The break did not abandon the generator halfway; "closed" is in the log. The sequence:

  1. The loop pulls 1 from the generator, which is now paused inside the try, one yield in.
  2. break starts leaving the loop, and on the way out, for-of calls the iterator's optional return() method.
  3. For a generator, return() resumes it just far enough to run its finally, so "closed" is pushed before the loop is fully gone.

Every other way out does the same. The return n inside firstOnly closes the generator too, which is the second "closed" in the final assertion, and a throw from the loop body closes it before the exception travels on, with the try/finally machinery itself measured on the errors and exceptions page in finally runs on every way out. That is what makes a generator a defensible place to hold a file handle or a database cursor: the cleanup runs whether the consumer finished, gave up, or failed.

a hand-written iterator gets no cleanup for free

Deno.test("a hand-written iterator gets no cleanup for free", () => {
const handles: string[] = [];
const leaky: Iterable<number> = {
[Symbol.iterator](): Iterator<number> {
handles.push("open");
let n = 0;
return { next: () => ({ value: n++, done: false }) };
},
};

for (const _ of leaky) break;

assertEquals(handles, ["open"]);
});
a hand-written iterator gets no cleanup for free ... ok (0ms)

Still open, and no error said so. The for-of looked for a return() method on the way out, found none, and moved on, because return() is optional and nothing warns you it is missing. The generator in the previous step got its finally run for free; this iterator recorded "open" and had no code path that could ever record a close. If you are writing an iterator by hand over something that needs closing, this is the part to get right. Or write a generator and let finally do it.

an endless source is ordinary

Deno.test("an endless source is ordinary", () => {
function* naturals(): Generator<number> {
let n = 1;
while (true) yield n++;
}

const taken: number[] = [];
for (const n of naturals()) {
taken.push(n);
if (taken.length === 3) break;
}
assertEquals(taken, [1, 2, 3]);

const [first, second] = naturals();
assertEquals([first, second], [1, 2]);
});
an endless source is ordinary ... ok (0ms)

Nothing in the protocol says a source ends, and naturals never does; the while (true) is the intentional infinite loop the loops page cleared with the linter in for is while with the moving parts at the top. Both consumers here survived it because both pull one value at a time: for-of with a break stopped after three, and the destructuring took two and never asked for a third.

The consumers that cannot stop are the ones whose job is to build a whole collection: spread, Array.from, new Set. Give one of those an endless source and the program hangs rather than failing, which is worth remembering when the iterable came from somewhere else in the codebase and its type, as the one-shot step showed, tells you nothing.

Array.from also accepts what is not iterable

Array.from has a second, older talent, and it muddies the water. Start with an object that has a length and numeric keys but no protocol:

Deno.test("Array.from also accepts what is not iterable", () => {
const arrayLike = { length: 2, 0: "sun", 1: "moon" };

assertEquals(Array.from(arrayLike as ArrayLike<string>), ["sun", "moon"]);

const copies = [...arrayLike];
assertEquals(copies, ["a", "b"]);
});
Check programs/iterables-and-iterators.test.ts
TS2488 [ERROR]: Type '{ length: number; 0: string; 1: string; }' must have a '[Symbol.iterator]()' method that returns an iterator.
const copies = [...arrayLike];
~~~~~~~~~
at file:///programs/iterables-and-iterators.test.ts:156:24

error: Type checking failed.

Array.from(arrayLike) is fine, and the spread of the very same object is refused before the program runs. The checker is telling the truth: Array.from takes two different kinds of thing, an iterable or an array-like, meaning an object with a length and numeric keys, and the spread only takes the first kind. Array-likes predate the protocol by years and survive in arguments and a few older DOM APIs. Suppose we do not believe the checker and claim our way past it, with the double cast from the any, unknown, and never page's as is a claim, not a conversion:

Deno.test("Array.from also accepts what is not iterable", () => {
const arrayLike = { length: 2, 0: "sun", 1: "moon" };

assertEquals(Array.from(arrayLike as ArrayLike<string>), ["sun", "moon"]);

const copies = [...(arrayLike as unknown as Iterable<string>)];
assertEquals(copies, ["a", "b"]);
});
Check programs/iterables-and-iterators.test.ts
running 10 tests from ./programs/iterables-and-iterators.test.ts
...
Array.from also accepts what is not iterable ... FAILED (0ms)

ERRORS

Array.from also accepts what is not iterable => ./programs/iterables-and-iterators.test.ts:151:11
error: TypeError: arrayLike is not iterable
const copies = [...(arrayLike as unknown as Iterable<string>)];
^

FAILURES

Array.from also accepts what is not iterable => ./programs/iterables-and-iterators.test.ts:151:11

FAILED | 9 passed | 1 failed (3ms)

error: Test failed

The claim changed the type and nothing else, so the spread asked for [Symbol.iterator](), found nothing, and threw. That pair of lines is the point of this step: Array.from succeeding tells you nothing about whether a value is iterable, because Array.from has a fallback and the spread does not. Pin the outcome with assertThrows:

Deno.test("Array.from also accepts what is not iterable", () => {
const arrayLike = { length: 2, 0: "sun", 1: "moon" };

assertEquals(Array.from(arrayLike as ArrayLike<string>), ["sun", "moon"]);

assertThrows(
() => [...(arrayLike as unknown as Iterable<string>)],
TypeError,
"arrayLike is not iterable",
);
});
Array.from also accepts what is not iterable ... ok (0ms)

an object is not iterable, and that will not change

Deno.test("an object is not iterable, and that will not change", () => {
const settings = { retries: 3 };

assertThrows(
() => {
// @ts-expect-error: Type '{ retries: number; }' must have a '[Symbol.iterator]()' method that returns an iterator.
[...settings];
},
TypeError,
"settings is not iterable",
);

assertEquals({ ...settings }, { retries: 3 });
assertEquals(Object.entries(settings), [["retries", 3]]);
});
an object is not iterable, and that will not change ... ok (0ms)

The pinned comment is the same TS2488 the previous step captured, and the runtime message is the same one the loops page captured for for-of in for-of demands the iterable protocol by name: refused before it runs, settings is not iterable while it runs. Two reasons this is a decision rather than an omission. Putting Symbol.iterator on Object.prototype would make every object in every program iterable, including ones whose keys are data and whose iteration would be meaningless. And there is no single right answer for what an object's values are: keys, values, or pairs are three defensible choices, which is exactly why Object.keys, Object.values, and Object.entries exist as three separate functions rather than one protocol.

Note the middle assertion. { ...settings } looks like a spread and is a different feature: object spread copies own enumerable properties and never touches the protocol, which is why it succeeds inside the very step that proves settings is not iterable. Same three dots, two unrelated mechanisms, and the one inside square brackets is the one that needs an iterable.

Iterable is the parameter type when all you do is loop

The names, first, since the steps above have been using them:

Deno.test("Iterable is the parameter type when all you do is loop", () => {
function total(values: Iterable<number>): number {
let sum = 0;
for (const value of values) sum += value;
return sum;
}

function* counted(): Generator<number> {
for (let n = 3; n > 0; n--) yield n;
}

assertStrictEquals(total([1, 2, 3]), 6);
assertStrictEquals(total(new Set([1, 2, 3])), 6);
assertStrictEquals(total(counted()), 6);
assertStrictEquals(total(new Map([["a", 1]]).values()), 1);
});
Iterable is the parameter type when all you do is loop ... ok (0ms)

One signature, four unrelated sources, no conversion at any call site. Taking Iterable<number> instead of number[] cost total nothing, because a single for-of pass is all it does, and it bought every caller the right to pass whatever they already have. The flip side came earlier: if a function loops twice, indexes, or needs a length, it should take T[] and say so in the signature, instead of discovering the requirement from a second loop that came back empty.

iterators have methods now

Deno.test("iterators have methods now", () => {
function* counted(): Generator<number> {
for (let n = 3; n > 0; n--) yield n;
}

assertEquals(counted().map((n) => n * 10).toArray(), [30, 20, 10]);
});
iterators have methods now ... ok (0ms)

Everything above treats an iterator as a thing you hand to a consumer and otherwise do not touch. That was the whole advice until recently, and it has changed: every built-in iterator now inherits map, filter, take, drop, reduce, toArray, and more from Iterator.prototype, and they are lazy. That is the subject of the iterator helpers page, worth reading next if you are here to transform data rather than to implement a container, and it makes the one-shot distinction from earlier considerably more consequential, because every chain of helpers ends in a single-use iterator.

A historical note, because older answers online turn on it. Compiling for-of over a non-array iterable down to old JavaScript required a flag named downlevelIteration, and a good deal of confusion came from that flag being off. Deno runs on a modern engine, so it never comes up here.

a generator method is a factory, not a one-shot

Deno.test("a generator method is a factory, not a one-shot", () => {
const deck = {
cards: ["ace", "king"],
*[Symbol.iterator](): Generator<string> {
for (const card of this.cards) yield card;
},
};

assertEquals([...deck], ["ace", "king"]);
assertEquals([...deck], ["ace", "king"]);
});
a generator method is a factory, not a one-shot ... ok (0ms)

Both spreads produce the full deck, which a bare generator could not do, and the difference is where the * sits. deck is not a generator; it is an object whose [Symbol.iterator] happens to be a generator method, so every consumer that asks for a walker runs the method again and gets a fresh generator. That restores the two-halves shape from the first step, with the position bookkeeping and the finally cleanup path for free. When you are writing a container, prefer this to a hand-written iterator: a bare generator is single-use, a generator method is a factory, and a factory is what the iterable half of the protocol was always asking for.

In practice