Executing Magento Indexer From Shell
In some situation when you want to re-index Magento catalog you will get an error message or reindex process will be broken. This problem is appearing when you have a lot of products in database and limited server resource. You can try to reindex from command line or shell.
In Magento there is folder with name “shell” in this folder you will find file with name “indexer.php” this is shell script for reindexing. You will ask yourself how to call this script, and take a look on my examples below and everything will be clear.
First of all you need to have access to linux commnad line or shell, and then go to your “MAGENTO_ROOT_FOLDER/shell“.
-jailshell-3.2$ pwd
/home/mcomerce
-jailshell-3.2$ ls
access-logs etc perl5 public_ftp ssl www
cpbackup-exclude.conf mail php.ini public_html tmp
-jailshell-3.2$ cd public_html/
-jailshell-3.2$ cd shell/
-jailshell-3.2$ ls
abstract.php compiler.php indexer.php log.php
-jailshell-3.2$Our first example is how to get help from “indexer.php” script, you need to execute next comand
“php -f indexer.php help” and you will get response as in example below.
$ php -f indexer.php help
--status <indexer> Show Indexer(s) Status
--mode <indexer> Show Indexer(s) Index Mode
--mode-realtime <indexer> Set index mode type "Update on Save"
--mode-manual <indexer> Set index mode type "Manual Update"
--reindex <indexer> Reindex Data
info Show allowed indexers
reindexall Reindex Data by all indexers
help This help
<indexer> Comma separated indexer codes or value "all" for all indexersYou can see that you can make reindexall but I don’t recommend that you execute full reindex in one step, beacuse you can cause MySql/Server overload. Reindexall calls 8 indexers in a row and process of reindexing can cause database LOCK on some database tables.
I recommend that you execute one by one indexer. You process will be faster and load of database will be smaller.
In Magento there are 8 indexers, you can get list of indexers if you call next command:
php -f indexer.php info
-jailshell-3.2$ php -f indexer.php info
catalog_product_attribute Product Attributes
catalog_product_price Product Prices
catalog_url Catalog Url Rewrites
catalog_product_flat Product Flat Data
catalog_category_flat Category Flat Data
catalog_category_product Category Products
catalogsearch_fulltext Catalog Search Index
cataloginventory_stock Stock status
-jailshell-3.2$For example if you want to reindex “catalog_url” you need to execute next command:
php -f indexer.php — -reindex catalog_url, below is example
-jailshell-3.2$ php -f indexer.php -- -reindex catalog_url
Catalog URL Rewrites index was rebuilt successfully
-jailshell-3.2$I recommend that you use indexer from shell because this process isn’t executing over web server/Apache and makes smaller impact on server resources :-).
Hope this was helpful. In case you need some help regarding Magento Development, we would be happy to help. Our team of experts would love to review your code and offer insights on improving your store. Our Magento Technical Audit is a great way to start – feel free to contact us!
One note: I had problem with php memory limit, error message: “Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 88 bytes)“. I increased memory size to 128Mb, I added next statement on beginning of script:ini_set(‘memory_limit’, ‘128M’); .
That’s all for now 😉
25 comments
Hi i am also facing issue in indexing category_urlrewrite
please check this screenshot – https://prnt.sc/jsee70
And i dont know what to write in php cli command to indexing from magmi product import it will be same script which i am running in cron job – php -f /public_html/shell/indexer.php –reindexall
please help me to get it solve..
Regards
Anand V.
Hi, I’m having a weird issue and I’m totally out of ideas how to fix it:
It doesn’t matter if I set the memory_limit to -1 or 1024MB in the php.ini file or in the shell/indexer.php file with ini_set, the indexer script always fails outputing the following error: PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 72 bytes) in /home/karanta/www/karanta.fr/lib/Zend/Db/Adapter/Abstract.php on line 691. 536870912 bytes is about 536MB right?
I’m sure I set the memory_limit value in the correct /etc/php5/cli/php.ini file. Also, when I check the return value of ini_set, I see the value I just set so it seems to work properly..
In the end I’m unable to complete the catalog_url reindex script. You guys at inchoo are my Magento heroes, have you ever ran into this kind of issue? Any suggestion where to look to fix this?
You should add the memory limit to your .htaccess file in the following format: 512M
The shell reindexer reads the .htaccess file
Also make sure the .htaccess file is encoded with lf line breaks.
Hi Inchooer,
i always had a question..
Who can explain to me the exact objective of each index? when i change the price i need to reindex only the price or everything? when do i need to reindex the url?
php indexer.php –reindex catalog_product_attribute
php indexer.php –reindex catalog_product_price
php indexer.php –reindex catalogsearch_fulltext
php indexer.php –reindex cataloginventory_stock
php indexer.php –reindex tag_summary
php indexer.php –reindex catalog_url
php indexer.php –reindex catalog_product_flat
php indexer.php –reindex catalog_category_flat
php indexer.php –reindex catalog_category_product
I use Indexer from the command line all the time. We run a multiwebserver environment. I’ll typically run it on my master webserver. I noticed on my slave webservers that it starts running the index job also.
I was evaluating the MySQL statements from the index and I noticed that the indexing queries were coming from all my servers.
Does anyone know a way around this? Maybe setting indexing to “Manual Mode”?
Hi, I’m having this same error:
The Error:
PHP Parse error: syntax error, unexpected T_VARIABLE, expecting T_FUNCTION
As Stefan said, I tried with
php5 indexer.php reindexall, but it says “Command not found”. Same thing forphp53 indexer.php reindexallI have a Linux shared hosting account at Go Daddy. I called them and asked about the problem, but they say that because of having a shared hosting account, those commands wont work, which doesn’t makes much sense to me.
Any ideas?
currently i am facing huge issue about re-indexing in magento 1.7,
i remove products from category by going to table
catalog_category_product and catalog_category_product_index and remove them from particular category,
what happens is when i remove them particular product disappears but when i run re-indexing again, products come back as re-indexing completes
i am dont know why this is happening to me.
please help me, it will be highly appreciated
First hit in Google on reindex catalog url shell””
And ….Exactly what I was looking for.
A perfect post. Thank you very much
The Error:
PHP Parse error: syntax error, unexpected T_VARIABLE, expecting T_FUNCTION in /var/www/lib/Varien/Db/Adapter/Pdo/Mysql.php on line 1554
is caused by a wrong php Version of your Shells PHP.
Use php5 instead:
php5 indexer.php reindexall
On siteground hosting with magento 1.7
command: /usr/local/bin/php shell/indexer.php reindexall
works like charm. Thank you
it fix my reindex problem of magento 1.7.0.2. it is the geatest post i have seen. i did search for the solution for a couple of weeks. and thinking to give it up. you save my site?s life. thanks and merry Christmas..
I Inchooer’s & everybody,
We have just released a little patch that optimize time processing when «Catalog Url Rewrites» is launched.
For exemple:
With 9500 references (SKU), shared (or not) between 900 categories / Sub-categories and with CMS pages, the number of generated URLs in the «Core_URL_Rewrite» table is about 80 000.
The re-index (launched manually or by CRON) lasts about 3h30.
After installing this patch, the re-index’s duration is about 1 minute.
Full article & Benchmark available @ : http://www.dnd.fr/2012/09/magento-patch-how-to-optimize-re-index-processing-time-for-url-rewrite/
We hope that this will help many people.
Your feedbacks are welcome 😉
Christophe
Hi, in which file did you put the script ini_set(‘memory_limit’, ’128M’); ?
I am running through the same memory problem, but increasing php memory limit in .htaccess and in php.ini didn’t solve my problem.
Is there any other file where this paramater must be edited?
thanks!
Thanks Domagoj,
This article was very helpful. If we update just the price in magento using magmi import, then which are the minimum ‘indexer options’ we need to execute so that the price is updated both on the front end product catalog/product details page and also in the ‘shopping cart’. I just did the reindex “-reindex catalog_product_price” but this updated only the product catalog and detail page, but during checkout it shows old price. Please advise.
I have the same problem with Solr. When I reindex from command line there is no results in Solr. When I use the admin panel reindex the results are shown. I think it is about rights issue, but i didn’t figure it out.
I found this post extremely useful and well written. I’ve been wanting a way to reindex from a shell script for a long time. This was exactly what I needed.
Thanks!
We are also facing similar issue as that of Roshan..When we reindexAll method , it does not index custom attributes created in Solr Index..However it does reindex all custom attribute data when we try to reindexing using magento admin panel..
I am using magento enterprise and SOLR search, my question is that, can we make cron job that re-index catalog search. I have used reindexAll method but that didnt work for me..Please help
Is it possible to call the indexer.php from the browser or via curl?
Because I’m on a hosting without shell access (and with shell_exec disabled).
When I try to do reindex from command line through
php indexer.php reindexall
getting following error.
PHP Parse error: syntax error, unexpected T_VARIABLE, expecting T_FUNCTION in /var/www/lib/Varien/Db/Adapter/Pdo/Mysql.php on line 1554
Due to reindex issue , products are not showing in the web page.
Kindly guide me to fix this issue.
Yes. As u said Ruiwant !. Great post. I was suffering from the same issue for more than 100,000 products in a store with huge amount of attributes. Thx! all.
Great post guy! I followed your instruction and make a sitemap script which can run through shell, but it’s so slow to generate it as long as run through browser. Do your guys have a better solution to generate while there are more then 100,000 products in stores.
To add to Anton’s response CLI scripts usually don’t have a time limit, so it’s more safe to run reindexing there as the process usually takes more than 30 seconds to finish
Would be nice to hear about other shell scripts for you. Thanks
You can reindex, clear (or enable or disable) the cache, and do much more with Wiz, the open-source Magento CLI: http://bit.ly/wiz-cli. You guys should check it out!
Note: almost 99% of hosting environments have different php.ini for php-cli so first you must ensure that your php-cli is meeting the magneto system requirements (all extensions needed, memory and time limits)