Executing Magento Indexer From Shell

Featured Image

In some situation when you want to re-index Magento catalog you will get an error message or reindex process will be broken. This problem is appearing when you have a lot of products in database and limited server resource. You can try to reindex from command line or shell.
In Magento there is folder with name “shell” in this folder you will find file with name “indexer.php” this is shell script for reindexing. You will ask yourself how to call this script, and take a look on my examples below and everything will be clear.

First of all you need to have access to linux commnad line or shell, and then go to your “MAGENTO_ROOT_FOLDER/shell“.

-jailshell-3.2$ pwd
/home/mcomerce
-jailshell-3.2$ ls
access-logs            etc   perl5    public_ftp   ssl  www
cpbackup-exclude.conf  mail  php.ini  public_html  tmp
-jailshell-3.2$ cd public_html/
-jailshell-3.2$ cd shell/
-jailshell-3.2$ ls
abstract.php  compiler.php  indexer.php  log.php
-jailshell-3.2$

Our first example is how to get help from “indexer.php” script, you need to execute next comand
php -f indexer.php help” and you will get response as in example below.

$ php -f indexer.php help
  --status <indexer>            Show Indexer(s) Status
  --mode <indexer>              Show Indexer(s) Index Mode
  --mode-realtime <indexer>     Set index mode type "Update on Save"
  --mode-manual <indexer>       Set index mode type "Manual Update"
  --reindex <indexer>           Reindex Data
  info                          Show allowed indexers
  reindexall                    Reindex Data by all indexers
  help                          This help
  <indexer>     Comma separated indexer codes or value "all" for all indexers

You can see that you can make reindexall but I don’t recommend that you execute full reindex in one step, beacuse you can cause MySql/Server overload. Reindexall calls 8 indexers in a row and process of reindexing can cause database LOCK on some database tables.

I recommend that you execute one by one indexer. You process will be faster and load of database will be smaller.

In Magento there are 8 indexers, you can get list of indexers if you call next command:
php -f indexer.php info

-jailshell-3.2$ php -f indexer.php info
catalog_product_attribute     Product Attributes
catalog_product_price         Product Prices
catalog_url                   Catalog Url Rewrites
catalog_product_flat          Product Flat Data
catalog_category_flat         Category Flat Data
catalog_category_product      Category Products
catalogsearch_fulltext        Catalog Search Index
cataloginventory_stock        Stock status
-jailshell-3.2$

For example if you want to reindex “catalog_url” you need to execute next command:
php -f indexer.php — -reindex catalog_url, below is example

-jailshell-3.2$ php -f indexer.php -- -reindex catalog_url
Catalog URL Rewrites index was rebuilt successfully
-jailshell-3.2$

I recommend that you use indexer from shell because this process isn’t executing over web server/Apache and makes smaller impact on server resources :-).

One note: I had problem with php memory limit, error message: “Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 88 bytes)“. I increased memory size to 128Mb, I added next statement on beginning of script:ini_set(‘memory_limit’, ’128M’); .
That’s all for now ;-)

21
Top

Care to rate this post?

Author

Domagoj Potkoc

Backend Developer

Domagoj is Magento Certified Developer who enjoys playing tennis after long hours in front of computer screen.

Other posts from this author

Discussion 21 Comments

Add Comment
  1. anton

    Note: almost 99% of hosting environments have different php.ini for php-cli so first you must ensure that your php-cli is meeting the magneto system requirements (all extensions needed, memory and time limits)

  2. You can reindex, clear (or enable or disable) the cache, and do much more with Wiz, the open-source Magento CLI: http://bit.ly/wiz-cli. You guys should check it out!

  3. To add to Anton’s response CLI scripts usually don’t have a time limit, so it’s more safe to run reindexing there as the process usually takes more than 30 seconds to finish

    Would be nice to hear about other shell scripts for you. Thanks

  4. Great post guy! I followed your instruction and make a sitemap script which can run through shell, but it’s so slow to generate it as long as run through browser. Do your guys have a better solution to generate while there are more then 100,000 products in stores.

  5. Amal Shashika

    Yes. As u said Ruiwant !. Great post. I was suffering from the same issue for more than 100,000 products in a store with huge amount of attributes. Thx! all.

  6. When I try to do reindex from command line through

    php indexer.php reindexall

    getting following error.

    PHP Parse error: syntax error, unexpected T_VARIABLE, expecting T_FUNCTION in /var/www/lib/Varien/Db/Adapter/Pdo/Mysql.php on line 1554

    Due to reindex issue , products are not showing in the web page.

    Kindly guide me to fix this issue.

  7. Michele

    Is it possible to call the indexer.php from the browser or via curl?
    Because I’m on a hosting without shell access (and with shell_exec disabled).

  8. I am using magento enterprise and SOLR search, my question is that, can we make cron job that re-index catalog search. I have used reindexAll method but that didnt work for me..Please help

  9. We are also facing similar issue as that of Roshan..When we reindexAll method , it does not index custom attributes created in Solr Index..However it does reindex all custom attribute data when we try to reindexing using magento admin panel..

  10. Timothy B

    I found this post extremely useful and well written. I’ve been wanting a way to reindex from a shell script for a long time. This was exactly what I needed.

    Thanks!

  11. cirus

    I have the same problem with Solr. When I reindex from command line there is no results in Solr. When I use the admin panel reindex the results are shown. I think it is about rights issue, but i didn’t figure it out.

  12. Dustin

    Thanks Domagoj,

    This article was very helpful. If we update just the price in magento using magmi import, then which are the minimum ‘indexer options’ we need to execute so that the price is updated both on the front end product catalog/product details page and also in the ‘shopping cart’. I just did the reindex “-reindex catalog_product_price” but this updated only the product catalog and detail page, but during checkout it shows old price. Please advise.

  13. Hi, in which file did you put the script ini_set(‘memory_limit’, ’128M’); ?
    I am running through the same memory problem, but increasing php memory limit in .htaccess and in php.ini didn’t solve my problem.
    Is there any other file where this paramater must be edited?
    thanks!

  14. I Inchooer’s & everybody,

    We have just released a little patch that optimize time processing when «Catalog Url Rewrites» is launched.

    For exemple:

    With 9500 references (SKU), shared (or not) between 900 categories / Sub-categories and with CMS pages, the number of generated URLs in the «Core_URL_Rewrite» table is about 80 000.

    The re-index (launched manually or by CRON) lasts about 3h30.

    After installing this patch, the re-index’s duration is about 1 minute.

    Full article & Benchmark available @ : http://www.dnd.fr/2012/09/magento-patch-how-to-optimize-re-index-processing-time-for-url-rewrite/

    We hope that this will help many people.

    Your feedbacks are welcome ;)

    Christophe

  15. alex

    it fix my reindex problem of magento 1.7.0.2. it is the geatest post i have seen. i did search for the solution for a couple of weeks. and thinking to give it up. you save my site?s life. thanks and merry Christmas..

  16. On siteground hosting with magento 1.7
    command: /usr/local/bin/php shell/indexer.php reindexall
    works like charm. Thank you

  17. The Error:
    PHP Parse error: syntax error, unexpected T_VARIABLE, expecting T_FUNCTION in /var/www/lib/Varien/Db/Adapter/Pdo/Mysql.php on line 1554

    is caused by a wrong php Version of your Shells PHP.
    Use php5 instead:

    php5 indexer.php reindexall

  18. First hit in Google on reindex catalog url shell”"

    And ….Exactly what I was looking for.
    A perfect post. Thank you very much

  19. currently i am facing huge issue about re-indexing in magento 1.7,

    i remove products from category by going to table
    catalog_category_product and catalog_category_product_index and remove them from particular category,

    what happens is when i remove them particular product disappears but when i run re-indexing again, products come back as re-indexing completes
    i am dont know why this is happening to me.
    please help me, it will be highly appreciated

  20. Andres Pinilla

    Hi, I’m having this same error:
    The Error:
    PHP Parse error: syntax error, unexpected T_VARIABLE, expecting T_FUNCTION

    As Stefan said, I tried with php5 indexer.php reindexall, but it says “Command not found”. Same thing for php53 indexer.php reindexall

    I have a Linux shared hosting account at Go Daddy. I called them and asked about the problem, but they say that because of having a shared hosting account, those commands wont work, which doesn’t makes much sense to me.
    Any ideas?

  21. Joe

    I use Indexer from the command line all the time. We run a multiwebserver environment. I’ll typically run it on my master webserver. I noticed on my slave webservers that it starts running the index job also.

    I was evaluating the MySQL statements from the index and I noticed that the indexing queries were coming from all my servers.

    Does anyone know a way around this? Maybe setting indexing to “Manual Mode”?

Add Your Comment

Please wrap all source codes with [code][/code] tags.
Top