Executing Magento Indexer From Shell

Featured Image

In some situation when you want to re-index Magento catalog you will get an error message or reindex process will be broken. This problem is appearing when you have a lot of products in database and limited server resource. You can try to reindex from command line or shell.
In Magento there is folder with name “shell” in this folder you will find file with name “indexer.php” this is shell script for reindexing. You will ask yourself how to call this script, and take a look on my examples below and everything will be clear.

First of all you need to have access to linux commnad line or shell, and then go to your “MAGENTO_ROOT_FOLDER/shell“.

-jailshell-3.2$ pwd
/home/mcomerce
-jailshell-3.2$ ls
access-logs            etc   perl5    public_ftp   ssl  www
cpbackup-exclude.conf  mail  php.ini  public_html  tmp
-jailshell-3.2$ cd public_html/
-jailshell-3.2$ cd shell/
-jailshell-3.2$ ls
abstract.php  compiler.php  indexer.php  log.php
-jailshell-3.2$

Our first example is how to get help from “indexer.php” script, you need to execute next comand
php -f indexer.php help” and you will get response as in example below.

$ php -f indexer.php help
  --status <indexer>            Show Indexer(s) Status
  --mode <indexer>              Show Indexer(s) Index Mode
  --mode-realtime <indexer>     Set index mode type "Update on Save"
  --mode-manual <indexer>       Set index mode type "Manual Update"
  --reindex <indexer>           Reindex Data
  info                          Show allowed indexers
  reindexall                    Reindex Data by all indexers
  help                          This help
 
  <indexer>     Comma separated indexer codes or value "all" for all indexers

You can see that you can make reindexall but I don’t recommend that you execute full reindex in one step, beacuse you can cause MySql/Server overload. Reindexall calls 8 indexers in a row and process of reindexing can cause database LOCK on some database tables.

I recommend that you execute one by one indexer. You process will be faster and load of database will be smaller.

In Magento there are 8 indexers, you can get list of indexers if you call next command:
php -f indexer.php info

-jailshell-3.2$ php -f indexer.php info
catalog_product_attribute     Product Attributes
catalog_product_price         Product Prices
catalog_url                   Catalog Url Rewrites
catalog_product_flat          Product Flat Data
catalog_category_flat         Category Flat Data
catalog_category_product      Category Products
catalogsearch_fulltext        Catalog Search Index
cataloginventory_stock        Stock status
-jailshell-3.2$

For example if you want to reindex “catalog_url” you need to execute next command:
php -f indexer.php — -reindex catalog_url, below is example

-jailshell-3.2$ php -f indexer.php -- -reindex catalog_url
Catalog URL Rewrites index was rebuilt successfully
-jailshell-3.2$

I recommend that you use indexer from shell because this process isn’t executing over web server/Apache and makes smaller impact on server resources :-).

One note: I had problem with php memory limit, error message: “Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 88 bytes)“. I increased memory size to 128Mb, I added next statement on beginning of script:ini_set(‘memory_limit’, ‘128M’); .
That’s all for now ;-)

Interested in hiring us?

Have a chat with us. You would be surprised how small changes can make your business even more successful.


About Domagoj Potkoc

Backend Developer

Domagoj is Magento Certified Developer who enjoys playing tennis after long hours in front of computer screen.

Read more posts by Domagoj / Visit Domagoj's profile

22 comments

  1. Hi Inchooer,

    i always had a question..

    Who can explain to me the exact objective of each index? when i change the price i need to reindex only the price or everything? when do i need to reindex the url?

    php indexer.php –reindex catalog_product_attribute
    php indexer.php –reindex catalog_product_price
    php indexer.php –reindex catalogsearch_fulltext
    php indexer.php –reindex cataloginventory_stock
    php indexer.php –reindex tag_summary
    php indexer.php –reindex catalog_url
    php indexer.php –reindex catalog_product_flat
    php indexer.php –reindex catalog_category_flat
    php indexer.php –reindex catalog_category_product

  2. I use Indexer from the command line all the time. We run a multiwebserver environment. I’ll typically run it on my master webserver. I noticed on my slave webservers that it starts running the index job also.

    I was evaluating the MySQL statements from the index and I noticed that the indexing queries were coming from all my servers.

    Does anyone know a way around this? Maybe setting indexing to “Manual Mode”?

  3. Hi, I’m having this same error:
    The Error:
    PHP Parse error: syntax error, unexpected T_VARIABLE, expecting T_FUNCTION

    As Stefan said, I tried with php5 indexer.php reindexall, but it says “Command not found”. Same thing for php53 indexer.php reindexall

    I have a Linux shared hosting account at Go Daddy. I called them and asked about the problem, but they say that because of having a shared hosting account, those commands wont work, which doesn’t makes much sense to me.
    Any ideas?

  4. currently i am facing huge issue about re-indexing in magento 1.7,

    i remove products from category by going to table
    catalog_category_product and catalog_category_product_index and remove them from particular category,

    what happens is when i remove them particular product disappears but when i run re-indexing again, products come back as re-indexing completes
    i am dont know why this is happening to me.
    please help me, it will be highly appreciated

  5. The Error:
    PHP Parse error: syntax error, unexpected T_VARIABLE, expecting T_FUNCTION in /var/www/lib/Varien/Db/Adapter/Pdo/Mysql.php on line 1554

    is caused by a wrong php Version of your Shells PHP.
    Use php5 instead:

    php5 indexer.php reindexall

  6. On siteground hosting with magento 1.7
    command: /usr/local/bin/php shell/indexer.php reindexall
    works like charm. Thank you

  7. it fix my reindex problem of magento 1.7.0.2. it is the geatest post i have seen. i did search for the solution for a couple of weeks. and thinking to give it up. you save my site?s life. thanks and merry Christmas..

  8. I Inchooer’s & everybody,

    We have just released a little patch that optimize time processing when «Catalog Url Rewrites» is launched.

    For exemple:

    With 9500 references (SKU), shared (or not) between 900 categories / Sub-categories and with CMS pages, the number of generated URLs in the «Core_URL_Rewrite» table is about 80 000.

    The re-index (launched manually or by CRON) lasts about 3h30.

    After installing this patch, the re-index’s duration is about 1 minute.

    Full article & Benchmark available @ : http://www.dnd.fr/2012/09/magento-patch-how-to-optimize-re-index-processing-time-for-url-rewrite/

    We hope that this will help many people.

    Your feedbacks are welcome ;)

    Christophe

  9. Hi, in which file did you put the script ini_set(‘memory_limit’, ’128M’); ?
    I am running through the same memory problem, but increasing php memory limit in .htaccess and in php.ini didn’t solve my problem.
    Is there any other file where this paramater must be edited?
    thanks!

  10. Thanks Domagoj,

    This article was very helpful. If we update just the price in magento using magmi import, then which are the minimum ‘indexer options’ we need to execute so that the price is updated both on the front end product catalog/product details page and also in the ‘shopping cart’. I just did the reindex “-reindex catalog_product_price” but this updated only the product catalog and detail page, but during checkout it shows old price. Please advise.

  11. I have the same problem with Solr. When I reindex from command line there is no results in Solr. When I use the admin panel reindex the results are shown. I think it is about rights issue, but i didn’t figure it out.

  12. I found this post extremely useful and well written. I’ve been wanting a way to reindex from a shell script for a long time. This was exactly what I needed.

    Thanks!

  13. We are also facing similar issue as that of Roshan..When we reindexAll method , it does not index custom attributes created in Solr Index..However it does reindex all custom attribute data when we try to reindexing using magento admin panel..

  14. I am using magento enterprise and SOLR search, my question is that, can we make cron job that re-index catalog search. I have used reindexAll method but that didnt work for me..Please help

  15. Is it possible to call the indexer.php from the browser or via curl?
    Because I’m on a hosting without shell access (and with shell_exec disabled).

  16. When I try to do reindex from command line through

    php indexer.php reindexall

    getting following error.

    PHP Parse error: syntax error, unexpected T_VARIABLE, expecting T_FUNCTION in /var/www/lib/Varien/Db/Adapter/Pdo/Mysql.php on line 1554

    Due to reindex issue , products are not showing in the web page.

    Kindly guide me to fix this issue.

  17. Yes. As u said Ruiwant !. Great post. I was suffering from the same issue for more than 100,000 products in a store with huge amount of attributes. Thx! all.

  18. Great post guy! I followed your instruction and make a sitemap script which can run through shell, but it’s so slow to generate it as long as run through browser. Do your guys have a better solution to generate while there are more then 100,000 products in stores.

  19. To add to Anton’s response CLI scripts usually don’t have a time limit, so it’s more safe to run reindexing there as the process usually takes more than 30 seconds to finish

    Would be nice to hear about other shell scripts for you. Thanks

  20. Note: almost 99% of hosting environments have different php.ini for php-cli so first you must ensure that your php-cli is meeting the magneto system requirements (all extensions needed, memory and time limits)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <blockquote cite=""> <code> <del datetime=""> <em> <strike> <strong>. You may use following syntax for source code: <pre><code>$current = "Inchoo";</code></pre>.