M2IF stands for (Magento 2 Import Framework). The idea behind M2IF is to provide a framework that allows developers to implement their own, custom Magento 2 import application. Basically M2IF provides a set of components, that provides the necessary functionality therefore. To make things more comfortable, it comes with a fully functional CLI sample application, based on Symfony, that should give you an idea, of how M2IF can be integrated in one of your own projects on the one hand, as well as PHAR to simply import products, on the other. Additionally M2IF is fully compatible with the Magento 2 standard product CSV format, which makes it possible to import all product CSV files that can be imported with the Magento 2 standard import functionaltiy as well.
This blog post will explain, how the console application can be used to import a bunch of products in a running Magento 2 CE example instance.
Be aware, that M2IF is fully functional but still under development! Therefore make sure, that you’ll backup your data before using it in your example Magento 2 installation.
Preparation
M2IF example CSV files are based on the Magento sample data, so it is necessary, if you want to run the example from this blog post, to have a running Magento 2.1.x CE instance WITH sample data on your system. Because the sample data installation of the CE on the commandline actually doesn’t work, the best approach would be to download the ZIP file from the Magento website and run the installation.
Installation
Assuming you’ve a running Magento 2 instance with sample data, and regarding HOW you want to use M2IF, the installation can be done in several ways. The easiest way, and the one we use in this blog post, would be to use the PHAR file, that can be downloaded from the Github release page, e. g. with wget by open a commandline, change to the root directory of your Magento installation and enter
wget https://github.com/techdivision/import-cli-simple/releases/download/1.0.0-beta62/import-cli-simple.phar \ -O bin/import-cli-simple.phar
If you’re on Mac OS X, you can install WGET with Homebrew. Now, as the PHAR is ready to use, try to invoke it by typing bin/import-cli-simple.phar. You should see the following output
M2IF - Simple Console Tool version 1.0.0-beta.62 Usage: command [options] [arguments] Options: -h, --help Display this help message -q, --quiet Do not output any message -V, --version Display this application version --ansi Force ANSI output --no-ansi Disable ANSI output -n, --no-interaction Do not ask any interactive question -v|vv|vvv, --verbose Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug Available commands: help Displays help for a command list Lists commands import import:attributes Imports attributes in the configured Magento 2 instance import:categories Imports categories in the configured Magento 2 instance import:clear:pid-file Clears the PID file from a previous import process, if it has not been cleaned up import:create:configuration-file Create's a configuration file from the given entity's template import:create:ok-file Create's the OK file for the CSV files of the configured source directory import:products Imports products in the configured Magento 2 instance import:products:inventory Imports product inventory in the configured Magento 2 instance import:products:price Imports product price in the configured Magento 2 instance
which means that the M2IF – Simple Console Tool is ready for use.
Usage
The import commands doesn’t implement any directory clean-up or archiving functionality, what means that you’ve to copy the the CSV files manually to the source directory. By default, the CSV files as well as the OK file are expected to be in the folder var/importexport below your Magento root directory.
Download the CSV files
So let’s start downloading the CSV files, containing the sample data, from our Github repository with
mkdir var/importexport \
&& wget https://github.com/techdivision/import-cli-simple/blob/master/projects/\
sample-data/ce/2.1.x/data/products/add-update/product-import.tar.gz?raw=true -O var/importexport/product-import.tar.gz \
&& tar xvfz var/importexport/product-import.tar.gz --directory var/importexport
This will create the directory var/importexport, downloads the archive with the CSV files product-import.tar.gz to the folder and extracts it there. Listing the directory’s content, should then look like
ls -l var/importexport/ total 4064 -rw-r--r--@ 1 you me 118015 18 Dez 08:51 product-import.tar.gz -rw-r--r-- 1 you me 451559 24 Sep 19:54 product-import_20161024-194026_01.csv -rw-r--r-- 1 you me 460347 24 Sep 19:54 product-import_20161024-194026_02.csv -rw-r--r-- 1 you me 472448 24 Sep 19:54 product-import_20161024-194026_03.csv -rw-r--r--@ 1 you me 568722 24 Sep 19:54 product-import_20161024-194026_04.csv
Create the .OK file
As the Magento default import functionality doesn’t use a OK file, it’ll be probably a good idea to give you a brief description what the OK file will be necessary for. As M2IF additionally supports bunches, whereas a bunch is a CSV file that contains only a part of the whole data that has to be imported, it is necessary to signal that all files for the import are available and the import process can be started now. As this will usually not be done manually on the commandline, but rather by something like a CRON job, this signal will be the OK file.
The import process only starts, when an OK file is available in the same directory where the CSV files are located. The name of the OK file MUST follow one of these naming conventions
- <IMPORT-DIRECTORY>/<PREFIX>.ok
- <IMPORT-DIRECTORY>/<PREFIX>_<FILENAME>.ok
- <IMPORT-DIRECTORY>/<PREFIX>_<FILENAME>_<COUNTER>.ok
which results in one of
- var/importexport/product-import.ok
- var/importexport/product-import_20161024-194026.ok
- var/importexport/product-import_20161024-194026_01.ok
when using our example CSV files. If you want to import a bunch, as we’ll do it in this blog post, the OK file MUST contain the name of the CSV files that have to be imported within the next iteration. In our case, the OF file should be named var/importexport/product-import.ok and MUST contain the following lines
product-import_20161024-194026_01.csv product-import_20161024-194026_02.csv product-import_20161024-194026_03.csv product-import_20161024-194026_04.csv
Creating the OK file can either be done manually or you may let the M2IF – Simple Console Tool create it for. Therefore invoke the following command
bin/import-cli-simple.phar import:create:ok-file
which should result in
Successfully written OK file var/importexport/product-import.ok
Importing the Bunches
Finally, when everything has been prepared, we’re ready to start importing the bunches, which are represented by the four CSV files in the var/importexport folder. Assumed, you’ve deleted the products before, the import will be very simple and can be done with the following command
bin/import-simple.phar import:products
which imports the four CSV file within one iteration. Round about 10 – 15 seconds later (depending on your hardware), you should see the following result
Now start import with serial 8e2ef8b0-5a78-40aa-8f4e-3e5e6ed65a1f [catalog_product => add-update] Successfully finished import with serial 8e2ef8b0-5a78-40aa-8f4e-3e5e6ed65a1f in 11,062358 s
The import has then been finished successfully and the import artefacts have been archived in the directory var/importexport/archive/8e2ef8b0-5a78-40aa-8f4e-3e5e6ed65a1f.tar.gz. As the actual version of the importer is NOT a Magento extension (we’re already working on it) do not forget to run the indexer with
bin/magento indexer:reindex && bin/magento cache:flush
to make sure the catalog will rendered as expected.
You may wonder what the reason for the bunch functionality is? Background for the bunch functionality is the attempt to parallelize the import process in the future to better utilize system resources.
Summing it Up
The M2IF, as a open source project, provides import functionality for both, the Community AND the Enterprise Edition. Actually it supports product, category and attribute import, each with the add/update, replace and the delete operations. Please checkout our Github repository for a complete list of features. Additionally there is a library that supports the Magic360 extension from a third party vendor. As we use M2IF in two of our actual projects (and will use it in our upcoming projects), functionality and stability will increase within the next month, which will result in the first stable version in Q1 2018.
Your feedback and your experiences with M2IF will be much appreciated. Also, we’ll be happy to see your PRs in our Github repository, if you’ve bugfixes or ideas for new functionality. This way, we hope that M2IF become a valuable part in the Magento ecosystem and the community as well.