Magento code library – Handling files and folders

Among most often custom coding tasks are usually those related to some import/export between Magento and other system. Usually existing store owners when switching from their current shop system to Magento want to transfer all of their products, orders, and most importantly customers.

Most of the time, standard Magento product/customer import/export is not sufficient for the task at hand. Limitations like file size upload and structure of original data require you to do a custom scripts that will handle the job of data transfer. Think of shops with several tens of thousands of orders, customers, products.

Luckily Magento comes with several nice classes that handle files and directory manipulation that might help you greatly. Surely all of this can be done with plain functional PHP, but hey why not impress your boss right?! 🙂

So, in the example below, I will show you how you can create a folder, fetch data via some external SFTP, parse/import, then parse/export it and finally push the data to some external SFTP.

First we will define some folder/file names on top of the script like shown below.

$baseDir = Mage::getBaseDir();
$varDir = $baseDir.DS.'var';
 
$timeOfImport = date('jmY_his');
 
$importReadyDir = $varDir.DS.'import_ready'.DS.$timeOfImport;
$exportReadyDir = $varDir.DS.'export_ready'.DS.$timeOfImport;
 
$_fileToImportRemote = '/home/ajzele/customers.txt';
$_fileToExportRemote = '/home/ajzele/customers_export.txt';
 
$_fileToImportBaseName = 'customers.txt';
$_fileToImportLocal = $importReadyDir.DS.'customers.txt';
$_fileToExportLocal = $exportReadyDir.DS.'customers_parsed.txt';

After this, lets separate our process to few steps. First we will grab the files from some SFTP and dump them locally to /var/import_ready/{{date(‘jmY_his’)}}/ folder.

$file = new Varien_Io_File();
 
//Create import_ready folder
$importReadyDirResult = $file->mkdir($importReadyDir);
 
if (!$importReadyDirResult) {
//Handle error
}
 
$sftpPickupFile = new Varien_Io_Sftp();
 
try {
$sftpPickupFile->open(
array(
'host' => 'some.server.com',
'username' => 'ajzele',
'password' => 'MyPass',
'timeout' => '10'
)
);
 
$_fileToImportRemoteTmp = $sftpPickupFile->read($_fileToImportRemote);
 
if(!$_fileToImportRemoteTmp) {
//Handle error
}
 
$sftpPickupFile->close();
 
if (!$file->write($_fileToImportLocal, $_fileToImportRemoteTmp)) {
//Handle error
}
 
} catch (Exception $e) {
//Handle error
}

Please note that the examples above are for ideal scenario with no errors in the process. Surely you would need more robust code to handle error scenarios.

At this point you should have your data pulled from SFTP and store locally in /var/import_ready/{{date(‘jmY_his’)}}/ folder. Where {{date(‘jmY_his’)}} will give you the exact date, hour, minute for folder name.

So, next step might be exporint some data to remote SFTP server? If so, here is a little example that demostrates how.

$flocal = new Varien_Io_File();
$flocal->open(array('path' => $importReadyDir));
$flocal->streamOpen('customers.txt', 'r');
 
while (false !== ($csvLine = $flocal->streamReadCsv())) {
 
//Parse the data and import it...
 
//Zend_Debug::dump($csvLine, '$csvLine');
/**
$csvLine array(4) {
[0] => string(13) "Branko Ajzele"
[1] => string(16) "ajzele@email.com"
[2] => string(2) "28"
[3] => string(25) "Sample address for Ajzele"
}
*/
}
 
//Now we do reverse, grab some data from Magento and upload it to SFTP
 
$data = '"Branko2 Ajzele", "ajzele@email.com", "28", "Sample address for Ajzele"
"Tomas2 Novoselic", "tomas.novoselic@email.com", "29", "Sample address for Tomas"';
 
$sftpDumpFile = new Varien_Io_Sftp();
 
try {
$sftpDumpFile->open(
array(
'host' => 'some.server.com',
'username' => 'ajzele',
'password' => 'MyPass',
'timeout' => '10'
)
);
 
//Make a local backup that will be send to SFTP
$file->mkdir($exportReadyDir);
$file->write($_fileToExportLocal, $data);
//Upload to SFTP
$_fileToExportRemoteTmp = $sftpDumpFile->write($_fileToExportRemote, $data);
 
} catch (Exception $e) {
echo $e->getMessage();
}

Enough with the examples. To conclude the things. There are some pretty decent classes in Varien library under Magento that enable you file and directory manipulation.

All you need to do sometimes is just poke around the existing library.