]]> Magento database heavily utilizes an Entity-Attribute-Value (EAV) data model. However, the cost of flexibility is often complexity. The process of manipulating EAV data in Magento is often more difficult than manipulating flat relational tables.

All Magento Models inherit from the Mage_Core_Model_Abstract. Difference between simple Model or an EAV Model is its Model Resource.

To build a model with proper collection object in Magento you need 4 things:

• model class
• resource class
• collection class
• install script (the one found under mymodule_setup folder of your extension)

To extend this little further there is one more file you would most likely need when building an EAV based model, the Setup.php which extends from Mage_Eav_Model_Entity_Setup. You would need to implement the getDefaultEntities() method in it as this is what’s called during the extension installation in Magento.

Imagine your module is called Phonebook and put into the /Inchoo namespace under the local code pool. Logically you would need a model class like User which represents single entry into the phonebook. With this in mind you already need a structure like:

• app/etc/modules/Inchoo_Phonebook.xml
• app/code/local/Inchoo/Phonebook/etc/config.xml
• app/code/local/Inchoo/Phonebook/Model/User.php
• app/code/local/Inchoo/Phonebook/Model/Resource/User.php
• app/code/local/Inchoo/Phonebook/Model/Resource/User/Collection.php
• app/code/local/Inchoo/Phonebook/Model/Resource/Setup.php
• app/code/local/Inchoo/Phonebook/sql/inchoo_phonebook_setup/install-1.0.0.0.php

Now that we outlined the file structure, let’s get down to it and see what the content of files should look like in order for it to work.

app/etc/modules/Inchoo_Phonebook.xml

app/code/local/Inchoo/Phonebook/etc/config.xml

app/code/local/Inchoo/Phonebook/Model/User.php

app/code/local/Inchoo/Phonebook/Model/Resource/User.php

app/code/local/Inchoo/Phonebook/Model/Resource/User/Collection.php

app/code/local/Inchoo/Phonebook/Model/Resource/Setup.php

app/code/local/Inchoo/Phonebook/sql/inchoo_phonebook_setup/install-1.0.0.0.php

For each attribute type like “varchar”, “text”, “int”… you need to create a corresponding table like inchoo_phonebook_user_entity_varchar, inchoo_phonebook_user_entity_text, inchoo_phonebook_user_entity_int. Study the install-1.0.0.0.php and the config.xml file to see how to get that one done.

Finally, you can test your model and it’s collection by running the following somewhere in your code:

And that’s it. Although simple, there is some typing to be done in order to get the EAV based model functional. Hope it helps someone.

Cheers.

]]> Greetings fellow developers! Today I’ll try to explain the principles of EAV system and Magento’s implementation of it. This is a tutorial / explenation for beginners, so please be patient and even try to draw the logic on a paper if you don’t understand it. A small notice before we start. I won’t use specific SQL query examples since this will be a simplified example of principles of the EAV.

What is EAV?

EAV stands for Entity, Attribute and Value. And that’s where we begin with our story.

Entity is somewhat of a “data item”. In Magento, those are for example Categories, Products and Customers.

Attribute is again a “data item”, but to differ it from the Entity, in Magento, attributes are for example: Title, Description, Price, etc. for each Product entity.

And Value for simplicity of explanation represents “real value” that is attached to Entity’s Attribute

And now, how does it work? (a.k.a. The Fun Part)

Part 1 – saving the data

Again, for the sake of simplicity, let’s use a Product in Magento for example.

If you’ve ever added a new product to Magento, you know that you’ve had to add some title, description and price to it (there are more attributes but we are using 3 for this example). When you saved your product, you used a total of 3 tables to save that information (a few more, but those are irrelevant for our example). First, a new Entity was added to DB, second, In Attribute table, you saved attributes (title, description and price) attached to that product, and finally to Value table, you saved 3 rows, attached to that product’s attributes. Here’s a simplified schema for those 3 tables:

Let’s say that our inserted product got the ID value of 1, our attributes got IDs from 1-3 and values where assigned to attributes accordingly.

Part 2 – getting the data

This is the easy part to understand. Retrieving the data from any EAV system begins at Entity. First, you get the Product entity’s (in our example 1)  ID, then, you get it’s attributes and values accordingly. That’s about it regarding retrieving the data from EAV system.

The conclusion

Well, I hope you understood the above example, in Magento’s EAV. I say Magento’s because there can be (and there are) different variations regarding the EAV’s complexity, but the principle is same for all of them.

Why is EAV used?

It’s used because of scalability. You can add almost anything into DB without changing it’s structure. That would be the main pron of it.

But everything that has a pron, has a con too.

Major problem with EAV systems is that each of them is much slower than a custom made solution because of SQL complexity. You need quite a few joins on DB just to retrieve one single Entity (or “data item”) in opposite to, let’s say, one select query on a custom made solution.

After this brief explanation. You should understand why Magento uses EAV system. It uses it because Magento is designed to be scalable, regardless of speed problems.

I’ve attached a PDF of Magento’s DB structure (v. 1.3), so that you can see the complexity (and simplicity) of it. And a wiki refference.

I hope this helped somebody, and that you now understand that once you comprehend EAV’s logic, it’s quite simple.

P.S. Before me, two of my colleagues wrote  articles that are closely related to the EAV discussions. Tomislav Bilic firstly introduced Magento MySQL database diagram back in 2008, while last year Branko Ajzele opened a discussion on how to Escape from EAV the Magento way.

]]> If you install new version of Magento 1.4, you can see that existing differences in structures of eav attributes. Earlier all data of EAV attributes were in table eav_attribute, now in new version eav_attribute table is smaller, beacuse a lot of fileds moved in new tables.

Structure of mysql table eav_attribute, Magento version 1.4:

Structure of mysql table eav_attribute, Magento version 1.3:

In new version of Magento, we have new table as: catalog_eav_attribute, customer_eav_attribute and so on…..

One important thing: the model Mage_Eav_Model_Entity_Setup has changes in new version of Magento. I have a problems with update attributes.

]]> One of the differences between Magento eCommerce platform and lets say WordPress, when looked from developer point of view, is “avoid raw queries” approach. For a web application, Magento is massive system. His database, although not so massive but surely breathtaking with around 220 tables forces you to use “eye candy” EAV model to do even simple things.

For instance, if I were to tell you that I want you to retrieve 5 simple products from database that have price in the range of 200-600USD you would most likely spent 1-4 hours trying to work out the query. In the end you would be lucky if you even get the query that wont break on next Magento upgrade. Let me just remind you on some of the things you need to be careful in example above.

Your query would have to take in consideration the at least the following states: is my product visibility such that it can be seen in Catalog and search result, is my product enabled or disabled, is is out of stock, which website and store view has it been assigned, is it assigned to only one category and that category is disabled by some chance, does it have special price assigned, does it have promotion rules assigned, does it have content (descriptions) for multilingual site assigned.

All of those and more are questions that one has to take into consideration when doing raw database queries. It makes no sense to create query that fetches product and all its appropriate info (price, quantity, description…) if that product is disabled or assigned to category that is disabled. For instance, if you need “featured” product functionality and you manually create raw query that extracts one “featured” product to lets say home page. There you create nice block holding all the necessary info of the product with a link for lets say “Add to cart” or “View product”. Clicking one of those two link would give you most likely 404 if the product is disable or any other condition that enables the product to be shown is disabled.

So basically, raw SQL queries in Magento are a “no no” for most of the time. Too much work and you never know what they will change in future upgrades. Which brings me to this new trend of theirs which I like to call “Escape from EAV”.

EAV, also know as Entity Attribute Value was suppose to bee (or is) the next big thing in OOP. Anyhow, Magento and his way of EAV have two huge downfalls called LACK OF DOCUMENTATION and WHAT WILL THEY CHANGE IN NEXT UPGRADE. The other day I was working on a Magento shop that had 22 000 products. Almost all of them were simple products, just about 600 were grouped. The store was initially installed on Magento 1.2 version. Site was pretty much 95% complete from both graphic and development perspective. Anyhow, Magento version 1.3.1 was released and on client demand we decide to to upgrade. Important thing to keep in mind is that in version 1.3 Magento introduce “change in philosophy” concerning the EAV model.

Altough EAV sounds great (and really is great for most of the time), it can really slow things down with all those JOINS executed on database. So the Magento team decided to do a little flat tabling. Basically we now have massive data duplication in MySQL where data is taken from various tables and copied int one, the flat table. Flat tables were introduced for both Products and categories, in regards to various website and store views.

Basically Magento has the power to “on the fly” create tables and do the “magical” copying of data from various tables to the flat tables. I assume they were looking for a faster way to “assemble” Product objects in Magento which in turn should boost the speed of collection object i grew so found about. I can live with duplicated data in database, I mean I am not the one writing them down. But let me get back to real world scenario.

As I mentioned, store I have been working on had more than 20 000 products. Due to limitations of both MySQL and PHP failed to do Rebuilt that you can find in System > Configuration > Cache management. This failure caused corrupted data in database. This manifested with “empty” attributes (attributes not showing up in Layered navigation). I resolved the issue by “re saving” all of the 20 000 product with custom script I wrote. After that i noticed the data in flat tables got rebuild as well. This of course took several hours. Magento EAV model seems OK on paper, but with store that hold’s large number of products it simply does not work, at least not in way in which their collection objects are built. Changes introduced in form of flat tables in versions 1.3 should, to some extent, improve work with stores that have large numbers of products. However this improvement seems to come in form of “copy all data from scattered tables to single flat table”.

Solutions like these are nightmare when it comes to upgrading the store that already has large number of products in. System is expected to handle part of the job transparently but it failed, leaving the client with corrupted database.

So, a word of advice for upgrading an existing Magento store: NEVER do it on live site. Let professional developer transfer live site and database to his dev machine, or make a copy on some sub folder on live site. Magento’s extreme out of the box feature rich capabilities, open source philosophy and good marketing have made it extremely popular but be careful, free is such a a loose term.

I am really anxious to see the future path of EAV vs Flat model in Magento.

PS. This topic was discussed on two more following articles from my colleagues. Tomislav Bilic firstly introduced Magento MySQL database diagram back in 2008, while Mladen Lotar expanded this intro and provided more details at his Magento’s database layout and it’s EAV structure article at 2010.

]]> If you worked with osCommerce, Zen Cart, CRE Loaded or any similar eCommerce platform before, you might find Magento database structure quite confusing when you see it for the first time. I advise you not to rush too much figuring out what is what by glancing through database. Try to spend first few hours getting familiar with some background. For purposes of flexibility, the Magento database heavily utilizes an Entity-Attribute-Value (EAV) data model. As is often the case, the cost of flexibility is complexity. Is there something in Magento that is simple from developers point of view?

Data manipulation in Magento is often more knowledge demanding than that typical use of traditional relational tables. Therefore, an understanding of EAV principles and how they have been modeled into Magento it is HIGHLY recommended before making changes to the Magento data or the Magento schema (Wikipedia: Entity-attribute-value_model). Varien has simplified the identification of EAV related tables with consistent naming conventions. Core EAV tables are prefixed with “EAV_”. Diagrams in this post contain a section labeled “EAV” which displays Magento’s core EAV tables and thier relationships to non-EAV tables.

Download Magento 1.1.6 MySQL database diagram (PDF)
Download Magento 1.3.2.4 MySQL database diagram (PDF)

Database diagrams and documents found in this post are intended to mirror the database schema as defined by Varien. Table relationships depicted in the diagrams represent only those relationships explicitly defined as Foreign Keys in the Magento database. Additional informal/undiagrammed table relationships may also exist, so when modifying the schema or directly manipulating data it is important to identify and evaluate possible changes to these tables as well (and the tables they relate to, and the tables they relate to…).

The author of Database Diagram is Gordon Goodwin, IT Consultant. You can see his info in the PDF.

PS. This topic was discussed on two more following articles from my colleagues. Mladen Lotar expanded this intro and provided more details at his Magento’s database layout and it’s EAV structure article at 2010, while in 2009. Branko Ajzele opened a discussion of how to Escape from EAV the Magento way.