<p>We ended up using this cli tool:
<a href="https://github.com/rsanchez/craft-cli" rel="nofollow noreferrer">https://github.com/rsanchez/craft-cli</a></p>

<p>We still had to tweak the php memory limit in the php config: <code>memory_limit 2048M</code></p>

<p>But eventually we managed to index 21.000+ images this way. </p>

CC BY-SA 3.0
2017-08-29T17:25:54.937Z
2017-08-29T17:25:54.937Z
Youniteus
<p>We uploaded a lot of images to the S3 bucket that is connected to our assets in Craft, but when we run "Update Asset Indexes" on the settings page, those images aren't added to the assets in Craft.</p>

<p>I'm pretty sure that the settings for the asset source is correct (images that we upload via Craft are working). And it doesn't seem to be related to a time-out, because we updated the php and nginx settings, and now it finishes after ~70s instead of 60s.</p>

<p>What are we doing wrong? 
Is it even possible to import images this way?</p>

CC BY-SA 3.0
2017-08-24T22:16:53.550Z
Anything in your `craft/storage/runtime/logs` files when you update the asset indexes? If you have your browser's console open when you run it, do you get any non-200 O.K. responses?
Brad Bell
CC BY-SA 3.0
2017-08-25T19:40:24.653Z
@BradBell Thanks for your comment. There was nothing in the craft logs, but the browser did returned a 502. I'm able to lower the timeout through nginx settings, but not increase it. So my current idea is that it's memory related. Maybe trying to import 21000+ images this way is to much. :o I'm thinking that we need to make a plugin to handle this amount of images.
Youniteus
CC BY-SA 3.0
2017-08-25T19:50:37.580Z
Just found this related question (https://craftcms.stackexchange.com/questions/6505/update-asset-index-in-batches) with a possible solution through a plugin (https://github.com/a-digital/assetindexer).
Youniteus
CC BY-SA 3.0
2017-08-27T11:23:31.433Z
Hmm that plugin isn't working to well. We're going to try to tweak with the craft code, and possibly create our own plugin.
Youniteus
<assets><php><s3><nginx><aws>
Update Asset Indexes for new files in s3 not working
# Update Asset Indexes for new files in s3 not working

(Asked by: username_0 on 2017-08-24)

We uploaded a lot of images to the S3 bucket that is connected to our assets in Craft, but when we run "Update Asset Indexes" on the settings page, those images aren't added to the assets in Craft.

I'm pretty sure that the settings for the asset source is correct (images that we upload via Craft are working). And it doesn't seem to be related to a time-out, because we updated the php and nginx settings, and now it finishes after ~70s instead of 60s.

What are we doing wrong?
Is it even possible to import images this way?

**Tags:** assets, php, s3, nginx, aws

> **Comments:**
> **username_1**: Anything in your `craft/storage/runtime/logs` files when you update the asset indexes? If you have your browser's console open when you run it, do you get any non-200 O.K. responses?  
> **username_0**: @BradBell Thanks for your comment. There was nothing in the craft logs, but the browser did returned a 502. I'm able to lower the timeout through nginx settings, but not increase it. So my current idea is that it's memory related. Maybe trying to import 21000+ images this way is to much. :o I'm thinking that we need to make a plugin to handle this amount of images.  
> **username_0**: Just found this related question (https://craftcms.stackexchange.com/questions/6505/update-asset-index-in-batches) with a possible solution through a plugin (https://github.com/a-digital/assetindexer).  
> **username_0**: Hmm that plugin isn't working to well. We're going to try to tweak with the craft code, and possibly create our own plugin.  

---

## 1 answers

### Answer by username_0 on 2017-08-29. Score: 0 | Accepted answer

We ended up using this cli tool:
<https://github.com/rsanchez/craft-cli>

We still had to tweak the php memory limit in the php config: `memory_limit 2048M`

But eventually we managed to index 21.000+ images this way.


---

