Magento 2: How to add new search engine like Solr or Elasticsearch

Magento 2: How to add new search engine like Solr or Elasticsearch

Magento 2 Community Edition comes with support for only MySQL search engine, but some projects require better or more adjustable search engine in order to increase sales or conversion rate.
In this situation we are implementing Solr or Elasticsearch search engine.

In this post we will create a skeleton code or rough example which will introduce main classes and methods by which we can implement additional search engine like Solr or Elasticsearch. If you take a look in Magento 2 admin you can find search engine configuration on next location: Stores -> Configuration -> Catalog -> Catalog Search and drop down “Search Engine”.

In drop-down list you will notice that you have only MySQL engine and our first step will be to add additonal option in this drop-down list with label “Solr”. So, let’s start.

As per usual, you need to create a Magento 2 module (I suppose that you already know this process but if you don’t, you can read the tutorial here). In your module in etc folder you need to create file di.xml with next xml code:

    <type name="Magento\Search\Model\Adminhtml\System\Config\Source\Engine">
        <arguments>
            <argument name="engines" xsi:type="array">
                <item name="solr" xsi:type="string">Solr</item>
            </argument>
        </arguments>
    </type>

With this xml code we added a new option to our drop-down list with the option name “Solr“. If you created it properly and cleaned Magento cache, you will be able to see it in the drop-down where there’ll be a new option “Solr”. If you see it then it means that you added it properly.

In the next step we will start with php classes which is in charge for indexing data to search server.

First of all, we should implement Engine class, put in di.xml next code:

    <type name="Magento\CatalogSearch\Model\ResourceModel\EngineProvider">
        <arguments>
            <argument name="engines" xsi:type="array">
                <item name="solr" xsi:type="string">Inchoo\Solr\Model\ResourceModel\Engine</item>
            </argument>
        </arguments>
    </type> 

You can see that we introduced our own Engine class for “Inchoo\Solr\Model\ResourceModel\Engine“. Engine class is in charge for preparing data before it goes to our indexerHandler class (last endpoint before solr server) and Engine class has to implement: \Magento\CatalogSearch\Model\ResourceModel\EngineInterface.

Interface class contains next four methods:
processAttributeValue prepare attribute value to store in solr index
getAllowedVisibility retrieve allowed visibility values for current engine
allowAdvancedIndex define if current search engine supports advanced index
prepareEntityIndex prepare index array as a string glued by separator

These methods are mandatory and have to be implemented in your Engine class. For better understanding you can check/compare logic in similar MySQL native class: Magento\CatalogSearch\Model\ResourceModel\Engine.

Our example of skeleton class is below:

<?php
namespace Inchoo\Solr\Model\ResourceModel;
 
use Magento\CatalogSearch\Model\ResourceModel\EngineInterface;
 
 
class Engine implements EngineInterface
{
 
    protected $catalogProductVisibility;
    private $indexScopeResolver;
 
    public function __construct(
        \Magento\Catalog\Model\Product\Visibility $catalogProductVisibility,
        \Magento\Framework\Indexer\ScopeResolver\IndexScopeResolver $indexScopeResolver
    ) {
        $this->catalogProductVisibility = $catalogProductVisibility;
        $this->indexScopeResolver = $indexScopeResolver;
    }
 
    public function getAllowedVisibility()
    {
        return $this->catalogProductVisibility->getVisibleInSiteIds();
    }
 
    public function allowAdvancedIndex()
    {
        return false;
    }
 
    public function processAttributeValue($attribute, $value)
    {
        return $value;
    }
 
    public function prepareEntityIndex($index, $separator = ' ')
    {
        return $index;
    }
 
    public function isAvailable()
    {
        return true;
    }
}

Next step is creating indexerHandler with name “Inchoo\Solr\Model\Indexer\IndexerHandler” which has to implement Magento\Framework\Indexer\SaveHandler\IndexerInterface.
For implemetation of IndexerHandler you should add next code in your di.xml file:

   <type name="Magento\CatalogSearch\Model\Indexer\IndexerHandlerFactory">
        <arguments>
            <argument name="handlers" xsi:type="array">
                <item name="solr" xsi:type="string">Inchoo\Solr\Model\Indexer\IndexerHandler</item>
            </argument>
        </arguments>
    </type>

If you open IndexerInterface you will see four methods which you have to implement:
saveIndex add entities data to index
deleteIndex remove entities data from index
cleanIndex remove all data from index
isAvailable define if engine is available (you can implement ping to solr server and check is it live).

Our example of IndexerHandler skeleton class is below:

<?php
namespace Inchoo\Solr\Model\Indexer;
 
use Magento\Eav\Model\Config;
use Magento\Framework\App\ResourceConnection;
use Magento\Framework\DB\Adapter\AdapterInterface;
use Magento\Framework\Indexer\SaveHandler\IndexerInterface;
use Magento\Framework\Indexer\IndexStructureInterface;
use Magento\Framework\Search\Request\Dimension;
use Magento\Framework\Search\Request\IndexScopeResolverInterface;
use Magento\Framework\Indexer\SaveHandler\Batch;
use Magento\Framework\Indexer\ScopeResolver\IndexScopeResolver;
 
class IndexerHandler implements IndexerInterface
{
    private $indexStructure;
 
    private $data;
 
    private $fields;
 
    private $resource;
 
    private $batch;
 
    private $eavConfig;
 
    private $batchSize;
 
    private $indexScopeResolver;
 
    public function __construct(
        Batch $batch,
        array $data,
        $batchSize = 50
    ) {
        $this->batch = $batch;
        $this->data = $data;
        $this->batchSize = $batchSize;
    }
 
    public function saveIndex($dimensions, \Traversable $documents)
    {
        foreach ($this->batch->getItems($documents, $this->batchSize) as $batchDocuments) {
 
        }
    }
 
    public function deleteIndex($dimensions, \Traversable $documents)
    {
        foreach ($this->batch->getItems($documents, $this->batchSize) as $batchDocuments) {
 
        }
    }
 
    public function cleanIndex($dimensions)
    {
 
    }
 
    public function isAvailable()
    {
        return true;
    }
}

In these methods you should implement Solr PHP client which will proceed listed operations to Solr server. Very often used is Solarium PHP client.

With this step we are ending with process of indexing data to search-server.

Now you can check are your indexer works with next command (before in set search engine to SOLR in magento admin):

php /bin/magento indexer:reindex catalogsearch_fulltext

In the next, last step we will explain how to implement a new search engine on the Magento 2 frontend. Also, we have to modify di.xml and add next code:

<type name="Magento\Search\Model\AdapterFactory">
        <arguments>
            <argument name="adapters" xsi:type="array">
                <item name="solr" xsi:type="string">Inchoo\Solr\SearchAdapter\Adapter</item>
            </argument>
        </arguments>
    </type>

Our new adapter is class Inchoo\Solr\SearchAdapter\Adapter. Adapter class should implement Magento\Framework\Search\AdapterInterface. In our adapter we have to implement method query – this method accepts query request and process it. Take a look of our example and everything will be more clear.

<?php
namespace Inchoo\Solr\SearchAdapter;
 
use Magento\Framework\Search\AdapterInterface;
use Magento\Framework\Search\RequestInterface;
use Magento\Framework\Search\Response\QueryResponse;
use Inchoo\Solr\SearchAdapter\Aggregation\Builder;
 
 
class Adapter implements AdapterInterface
{
    protected $responseFactory;
 
    protected $connectionManager;
 
    protected $aggregationBuilder;
 
    public function __construct(
        ResponseFactory $responseFactory,
        Builder $aggregationBuilder,
        ConnectionManager $connectionManager
    ) {
        $this->responseFactory = $responseFactory;
        $this->aggregationBuilder = $aggregationBuilder;
        $this->connectionManager = $connectionManager;
 
    }
 
    /**
     * @param RequestInterface $request
     * @return QueryResponse
     */
    public function query(RequestInterface $request)
    {
        $client = $this->getConnection();
        $documents = [];
 
        $documents[1007] = array('entity_id'=>'1007', 'score'=>46.055);
        $documents[1031] = array('entity_id'=>'1031', 'score'=>45.055);
        $documents[1120] = array('entity_id'=>'1120', 'score'=>44.055);
 
        $aggregations = $this->aggregationBuilder->build($request, $documents);
 
        $response = [
            'documents' => $documents,
            'aggregations' => $aggregations,
        ];
        return $this->responseFactory->create($response);
    }
 
    public function getConnection(){
        return $this->connectionManager->getConnection();
    }
}

In our demo adapter class we hard coded product entity_ids: 1007, 1031, 1120 from our database product-ids, only for testing purpose. If you want to dig deeper I suggest that you examine logic how MySQL native adapter works.

With this step we are ending our example. Even though things seem pretty complicated, when you start working, everything will be fine. I hope that you will enjoy the coding of your new search engine for Magneto 2.

You made it all the way down here so you must have enjoyed this post! You may also like:

What’s New in Magento 2.4? Features and Benefits for Merchants Explained Antonio Gotovac
Antonio Gotovac, | 1

What’s New in Magento 2.4? Features and Benefits for Merchants Explained

Magento 2 Customer Groups Configuration Josip Kovacevic
Josip Kovacevic, | 2

Magento 2 Customer Groups Configuration

How did we standardize Magento 2 frontend development? Ljiljana Milkovic
Ljiljana Milkovic, | 8

How did we standardize Magento 2 frontend development?

12 comments

  1. Catalog Search indexer process unknown error:
    illegal_argument_exception: Illegal dynamic template parameter: [match_mapping]

  2. [ReflectionException]
    Class Inchoo\Solr\SearchAdapter\ResponseFactory does not exist
    [ReflectionException]
    Class Inchoo\Solr\SearchAdapter\ResponseFactory does not exist
    [RuntimeException]
    Source class “\Inchoo\Solr\SearchAdapter\Response” for “Inchoo\Solr\SearchAdapter\ResponseFactory” generation does
    not exist.

    Any helps please?

  3. Hello,
    Thank for the tutorial. I created a module like above but magento is not generating the file files for this modules. I am getting this error:
    \Solr\SearchAdapter\ResponseFactory” generation does not exist.

    How can I solve this issue?

    1. This class is not created in this tutorial :
      Inchoo\Solr\SearchAdapter\Aggregation\Builder;

  4. For a complete integration of ElasticSearch with Magento CE, you can also take a look at ElasticSuite which comes with much more than a simple search engine :

    Last but not least, it is Open Source.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <blockquote cite=""> <code> <del datetime=""> <em> <s> <strike> <strong>. You may use following syntax for source code: <pre><code>$current = "Inchoo";</code></pre>.

Tell us about your project

Drop us a line. We'd love to know more about your project.