Fetch all pages with Octokit pagination

A quick one today as I constantly need to fetch more than 100 results from the GitHub API using Octokit and always end up digging in to an old project to copy out this code snippet.

js
const repos = await octokit.paginate(
  octokit.repos.listForAuthenticatedUser,
  {
    per_page: 100,
  },
  (response) => response.data
);

This uses the octokit.paginate helper that detects if the response has a Link header, and if so it will automatically fetch the next page.

The first argument provided is the URL to fetch. This can be an Octokit route definition (such as octokit.repos.listForAuthenticatedUser), or it can be a string (in this case GET /user/repos, but it could also be a parameterised URL such as GET /repos/:owner/:repo).

The second argument is the list of parameters to provide. If the name matches a parameterised field in the URL (such as :repo) it will be used in that URL segment, otherwise it will added as a GET parameter.

Finally, and most importantly, is the callback in the third argument. This callback will be called for every page of results. In the example above we return response.data, which discards the response headers and makes repos an array of objects.

You may also manipulate the results in this callback to only return the repositories that you need. For example, here’s how to ignore archived repos:

js
const repos = await octokit.paginate(
  octokit.repos.listForAuthenticatedUser,
  {
    per_page: 100,
  },
  (response) => response.data.filter((r) => !r.archived)
);

This callback also accepts a second done parameter which can stop Octokit from fetching the next page. Imagine that we want to cap the list of repos at 250 repositories. You can do it like so:

js
let repoCount = 0;
const repos = await octokit.paginate(
  octokit.repos.listForUser,
  {
    username: "mheap",
    per_page: 100,
  },
  (response, done) => {
    repoCount += response.data.length;
    if (repoCount >= 250) {
      done();
    }
    return response.data;
  }
);

Note how we call done() if we have enough repositories returned to stop Octokit fetching the next page.

In this example, you could end up with 300 repositories as it fetches in blocks of 100. To ensure that you only get 250 maximum, you could change per_page to 50 or slice response.data in the callback.