<p>First, you'll want to make sure you're on the latest FeedMe release as a bug around this was recently fixed: <a href="https://github.com/craftcms/feed-me/issues/694" rel="nofollow noreferrer">https://github.com/craftcms/feed-me/issues/694</a></p>
<p>You might also need a plugin that hooks into the <code>EVENT_AFTER_FETCH_FEED</code> event to do some offset transformation based on the tokens passed in.</p>

CC BY-SA 4.0
2020-10-09T03:15:32.827Z
2020-10-09T03:15:32.827Z
Brad Bell
<p>I'm the one who posted that bug report regarding cursor pagination.  I just wanted to share a simple version of the code I'm using in case it's useful for anyone. I'm using Airtable so you can see I'm only running the function if the feed source is an Airtable URL.</p>
<p>For reference, I'm getting an offset value like <code>abcdefg</code> from the API and I'm changing it to <code>https://my-api-url.com/feed1/abcdefg</code> to get the next page of results.</p>
<p>You can run this from a module like the one you can generate at <a href="https://pluginfactory.io/" rel="nofollow noreferrer">https://pluginfactory.io/</a></p>
<pre><code>use craft\feedme\events\FeedDataEvent;
use craft\feedme\services\DataTypes;
use yii\base\Event;


class Module extends \yii\base\Module
{
    public function init()
    {
      Craft::setAlias('@modules', __DIR__);
      if (Craft::$app-&gt;getRequest()-&gt;getIsConsoleRequest()) {
          $this-&gt;controllerNamespace = 'modules\\console\\controllers';
      } else {
          $this-&gt;controllerNamespace = 'modules\\controllers';
      }

      parent::init();

      /* Changes the Airtable API pagination key on the fly */

      Event::on(DataTypes::class, DataTypes::EVENT_AFTER_FETCH_FEED, function(FeedDataEvent $event) {
        $response = $event-&gt;response;
        $feedUrl = $event-&gt;url;
        
        /* Only run if this is an Airtable feed with a response */
        if ( strpos($feedUrl, &quot;airtable&quot;) &amp;&amp; isset($response['data']) ) : 
          $data = json_decode($response['data']);
          
          /* We only need to update the event if there's an offset key */
          if ( isset($data-&gt;offset) ) {

            /* Break the URL up into it's components */
            $url = parse_url($feedUrl);
            $baseUrl = $url['scheme'] . '://' . $url['host'] . $url['path'];
            parse_str($url['query'], $params);

            /* Update the current query with the new offset value */
            $params['offset'] = $data-&gt;offset;

            /* Rebuild the pagination URL */
            $data-&gt;offset = $baseUrl . '?' . http_build_query($params, $enc_type = PHP_QUERY_RFC3986);

            /* Return the data to the event */
            $response['data'] = json_encode($data);
            $event-&gt;response = $response;
          }
        endif;
      });
    }
}
</code></pre>

CC BY-SA 4.0
2020-10-15T04:28:20.630Z
Thanks for sharing!
Brad Bell
CC BY-SA 4.0
2021-09-23T18:02:59.353Z
I just want to say how much I appreciate this! It's saved me hours of time.
dmatthams
CC BY-SA 4.0
2021-09-23T22:56:56.723Z
@dmatthams - Awesome, glad to hear it!
Dalton Rooney
CC BY-SA 4.0
2020-10-13T20:17:39.473Z
2020-10-13T20:17:39.473Z
Dalton Rooney
<p>Has anyone used a pagination token when paginating Feedme? I currently use Firestore and they don't use a full pagination URL but a token you append onto your next URL.</p>
<p>Initial URL
<code>https://firestore.googleapis.com/v1/projects/YOUR_PROJECT_ID/databases/(default)/documents/cities/LA</code></p>
<p>Pagination URL
<code>https://firestore.googleapis.com/v1/projects/YOUR_PROJECT_ID/databases/(default)/documents/cities/LA?pageToken=Ab7e2991jej91od727</code></p>
<p>In the Response</p>
<pre><code>&quot;nextPageToken&quot;: &quot;AFTOeJwxKBJYSXXPpKyhOmB4_-SifKCDzUH0mhJchn1u5hknl3iutUKaPez8OVyR99PZM2014ZU1EpZt4GnUHwHBnwM0P1B9Z-yDhTyqTrmsVt4SIffOXCyxq1Yl7iZQJ0c1ltGHpuV9KWgUH6mvl6rHdCA&quot;
</code></pre>

<plugin-feedme>
Feedme Pagination by Token
# Feedme Pagination by Token

(Asked by: username_0 on 2020-10-07)

Has anyone used a pagination token when paginating Feedme? I currently use Firestore and they don't use a full pagination URL but a token you append onto your next URL.
Initial URL
https://firestore.googleapis.com/v1/projects/YOUR_PROJECT_ID/databases/(default)/documents/cities/LA
Pagination URL
https://firestore.googleapis.com/v1/projects/YOUR_PROJECT_ID/databases/(default)/documents/cities/LA?pageToken=Ab7e2991jej91od727
In the Response
"nextPageToken": "AFTOeJwxKBJYSXXPpKyhOmB4_-SifKCDzUH0mhJchn1u5hknl3iutUKaPez8OVyR99PZM2014ZU1EpZt4GnUHwHBnwM0P1B9Z-yDhTyqTrmsVt4SIffOXCyxq1Yl7iZQJ0c1ltGHpuV9KWgUH6mvl6rHdCA"

**Tags:** plugin-feedme


---

## 2 answers

### Answer by username_2 on 2020-10-13. Score: 4 | Accepted answer

I'm the one who posted that bug report regarding cursor pagination.  I just wanted to share a simple version of the code I'm using in case it's useful for anyone. I'm using Airtable so you can see I'm only running the function if the feed source is an Airtable URL.
For reference, I'm getting an offset value like abcdefg from the API and I'm changing it to https://my-api-url.com/feed1/abcdefg to get the next page of results.
You can run this from a module like the one you can generate at https://pluginfactory.io/
use craft\feedme\events\FeedDataEvent;
use craft\feedme\services\DataTypes;
use yii\base\Event;


class Module extends \yii\base\Module
{
    public function init()
    {
      Craft::setAlias('@modules', __DIR__);
      if (Craft::$app->getRequest()->getIsConsoleRequest()) {
          $this->controllerNamespace = 'modules\\console\\controllers';
      } else {
          $this->controllerNamespace = 'modules\\controllers';
      }

      parent::init();

      /* Changes the Airtable API pagination key on the fly */

      Event::on(DataTypes::class, DataTypes::EVENT_AFTER_FETCH_FEED, function(FeedDataEvent $event) {
        $response = $event->response;
        $feedUrl = $event->url;
        
        /* Only run if this is an Airtable feed with a response */
        if ( strpos($feedUrl, "airtable") && isset($response['data']) ) : 
          $data = json_decode($response['data']);
          
          /* We only need to update the event if there's an offset key */
          if ( isset($data->offset) ) {

            /* Break the URL up into it's components */
            $url = parse_url($feedUrl);
            $baseUrl = $url['scheme'] . '://' . $url['host'] . $url['path'];
            parse_str($url['query'], $params);

            /* Update the current query with the new offset value */
            $params['offset'] = $data->offset;

            /* Rebuild the pagination URL */
            $data->offset = $baseUrl . '?' . http_build_query($params, $enc_type = PHP_QUERY_RFC3986);

            /* Return the data to the event */
            $response['data'] = json_encode($data);
            $event->response = $response;
          }
        endif;
      });
    }
}

> **Comments:**
> **username_1**: Thanks for sharing!  
> **username_3**: I just want to say how much I appreciate this! It's saved me hours of time.  
> **username_2**: @username_3 - Awesome, glad to hear it!  

---

### Answer by username_1 on 2020-10-09. Score: 2

First, you'll want to make sure you're on the latest FeedMe release as a bug around this was recently fixed: <https://github.com/craftcms/feed-me/issues/694>

You might also need a plugin that hooks into the `EVENT_AFTER_FETCH_FEED` event to do some offset transformation based on the tokens passed in.


---

