Every year, we had an adventcalendar on webguys.de. Checkout the posts from the last years 2016, 2014 and 2013. Due to the fact that webguys.de is a blog by three German developers and this started in the German community, many of the posts are in German. But more and more of the new posts are in English, so have a look! A big shout out goes to Tobias,who took care of the calendar in the last years. We nearly forgot it this year. But as always, the Magento community stands together in an emergency, when the distress call happens. The list of posts is not complete yet, so if you have a cool idea please send an email to Fabian.
It is a pleasure for us and especially me that you found the way to this year’s advent calender. Please come back every day, and check out the other posts as well!
But now right into the topic.
Caching is not the solution!
I think we can agree that magento is too slow. Don’t get me wrong, I love magento (especially version 1)! And back in the days (like 2 years ago ;-)) it was kind of fast enough. It was ok to wait a second or two for your content. But today, our customers expect to see the page before they even click the link. The fastest magento I heard about without full page cache (FPC) is 200-300ms, and this was already customised and most of the EAV stuff was replaced with a NoSQL database.
Most of us, use a full page cache to speed up your site. And if this works for you, you can stop reading here. You have a solution which works! What more can one expect. But many of us are unhappy with their FPCs, for a couple of reasons:
- Caching is complex. One of the easier, yet compelling examples is that if you disable a product, you not only need to refresh the listing page with the product, but every page after this page, because every product moves one position forward.
- Caching works great in many read-only environments. This means if you have a big team of content writers, price or inventory updates or something alike, your cache is invalidated all the time. You can decide that you don’t invalidate the cache and serve old content. But there is no need to make this tradeoff.
- If you decide to serve old content or your invalidation is not perfect you have inconsistencies.
- and if your store is not powerful enough a flush kills your complete site.
Lizards and Pumpkins generates a view-optimized format of your data, right after the data changed. This way the data are always up to date and blazingly fast at the user.
Step by step or only what you need
Lizards and Pumpkins sits right before your store. So every request runs through ?&? and it decides whether it processes the request or not. If the request is not processed by Lizards & Pumpkins, Magento gets its chance. If it can’t process it either, we get the our hated 404 error.
While importing all product data, Lizards & Pumpkins is building an URL storage to easily decide who answers the request. ?&? can currently take care of three different page types. “Can” means it doesn’t have to.
- Categories (or as we call it: product lists)
- Product detail pages
- Product search results
Depending on your store, you decide what you need.
You have this one category which is full of configurable products and loads to death? Then import all products of this category into Lizards & Pumpkins and serve this one category from Lizards and don’t touch the rest.
Your product pages and categories are blazing fast, but your search is horribly slow and the results are even worse? Replace the search with Lizards & Pumpkins and stay with your catalog!
You update your product prices all the time but show them only on product detail pages? Your FPC is working fine for product listings and the query cache solves performance issues with the search? Serve the product detail pages by Lizards & Pumpkins and make sure that prices and stock data are always up to date.
Whatever your problem with your catalog is, Lizards & Pumpkins can solve it, without breaking functionality with the legacy system. And when you started using it, you might find the other pages too slow as well, so move them step by step.
Use tech you already know
Lizards & Pumpkins is built upon PHP. The hard dependencies are: PHP7, php7.0-intl and php7.0-xml. End of list.
Theoretically you can run Lizards & Pumpkins with this setup, even in production, but we highly recommend to at least use a non-php search engine. ?&? needs three systems to help it doing its job: a search engine, a key value storage and a queue. A few implementations already exist, like Solr, Elasticssearch, Redis and ZeroMQ. But staying with the technology you already know is a cheap entry. You already use Amazon’s DynamoDB, Google Cloud’s Pub/Sub or CouchDB? Stay with it! Implementations for the Queue and the Key value storage are done in hours to days. The search engine takes a bit longer. But if you already has the knowledge to maintain the technology the investment is worth it.
Open Source
BSD-3clause. This means, no one can take the code from you. You are free to use it. Do your implementations. There is no obligation to give back in form of open sourcing your implementations, e.g. for a new key-value storage, although we highly encourage you. Beside the fact, that it is very good for your karma, you have more people testing your implementation and maintaining it, so win-win for all parties.
First step away from your legacy system
I love Magento, I really do. But there are edge cases were Magento is simply not the right system. I once had a customer who sold trainings which consist of trainings which consist of trainings. So levels and levels of recursion. Magento was not able to map this. And if you use other functionality, like price rules, wishlist, reviews and such, Magento might be the right choice. If you don’t, you have a huge dependency you can’t get rid off and built all your stuff into it.
You already did this? And you are at a point were your current system, be it Magento, Oxid or something else can’t handle your current and upcoming business needs? Then maybe it is an idea to do the first step away from your current system by replacing it, step by step.