Installation - Overview

• add the product attributes
  
• unzip the extension
  
• delete data sources we don’t need
  
• fix for using Aitoc_Permissions
  
• configure the extension
  
• customise the templates
  

If the module configuration section results in a 404 error then please try logging out and then in again from Magento admin.

A data source can be turned on by changing “Scheduled Import Enabled?” to “Yes”.

Installation - G3D ACP Specific Details

• set the ACP username, password and company ref in System / Configuration / Auto Import / G3D
  
• a top level category called “Directory” will be created, and the categories which that user is allowed to see will be imported under “Directory”
  
• in the configuration section, you also need to provide the company ID (GUID) for each supplier name, so that the 3D URLs can be correctly formed.  In the box, enter each company ID and supplier name (taken from ACP product info) separated by a comma, one per line, like this:
  
• 13555, Alexandra
  
• 12345, GBN
  
• 11111, Keramikos
  
• If a product is found without a matching supplier name, the import will stop and an error message will be sent to a developer
  

Installation - Ralawise Specific Details

• you can edit the grouped product detail template using the file app/design/frontend/base/default/templates/g3dautoimport/ralawise/grouped.phtml - copy that into your theme folder in the usual way, and make your changes there
  
• you can change when the import is allowed to run by editing app/code/local/Gateway3D/AutoImport/etc/config.xml - at the bottom is the cron expression for when it can run.  A full Ralawise import takes between 3 and 8 hours depending on the server.
  
• use a cron script to make the indexer run once per day, and not at the same time as the import.  eg. set the import to run between 0:00 and 5:59, and the indexer to run at 6:30am
  
• see section 11 below - products will not import until you configure ‘magmi’ with your database details
  
• if you are not using the Ralawise API, you need to manually put the catalogue files in the var/import/ folder - see below
  
• if you are using the Ralawise API, you need to made a little code change to switch it on - see below
  

Installation of the Extension

1. This extension is compatible with Aitoc Advanced Permissions, but after installing you need to change something:
   
1. In the file app/code/local/Gateway3D/AutoImport/etc/config.xml
   
2. find the ‘rewrite’ lines under blocks/g3dautoimport/adminhtml
   
3. remove the lines in there containing ‘catalog_product_edit’ and ‘catalog_category_edit_form’
   
4. flush the cache, if necessary
   
2. Add the custom attributes for products.  These are done manually so that you can choose how they are set up.  Add them in Catalog / Attributes then add them to the Default attribute set. All should be in scope ‘Global’.
   
1. Sourcing City:
   
1. add a text field with ID “sourcing_city_price_range”, not required
   
2. add a dropdown field with ID “sourcing_city_express”, not required
   
3. add a Yes/No field with ID “sourcing_city_britishmade”, not required
   
4. add a text field with ID “sourcing_city_colours”, not required
   
5. add a text field with ID “sourcing_city_advanced_search”, not required
   
6. add a multi-select field with ID “sourcing_city_colour_select”, not required, with use for layered navigation = yes
   
2. Ralawise:
   
1. add a text field with ID “ralawise_page”, not required
   
2. add a drop-down field with ID “ralawise_colour”, not required
   
3. add a drop-down field with ID “ralawise_size”, not required
   
4. add a drop-down field with ID “ralawise_brand”, not required
   
3. G3D ACP
   
1. (if 3D products required) install the Personalise-iT extension, which will put in all the required attributes
   
3. Unzip the Gateway3D_AutoImport extension in the root folder
   
4. Navigate to the app/code/local/Gateway3D/AutoImport/Model/Datasource folder
   
5. Delete the php file of any data source you want to HIDE in the configuration area, EXCEPT for Abstract.php - leave that one!   (NOTE - even if you don’t hide them, you can still disable them, so this is purely about preventing customers from receiving particular sources)
   
6. Make sure the Magento cron job is set up as part of your normal Magento installation, to run at least every 3 minutes - preferably every 1 minute.
   
7. Verify that it has installed by navigating to Catalog / Gateway3D AutoImport / Pricing Rules - if this screen exists, it is installed.
   
8. Logged in as the website user, create the folder /media/catalog/category/resized if not already existing
   
9. Logged in as the website user, create the folder /media/import if not already existing
   
10. In System/Configuration, set the following:
    
1. Catalog / Catalog / Price / Price Scope - if using multiple website pricing, set this to Website
   
2. Gateway3D / Auto Product Import - be sure to enable the data sources you want to use on this site.
   
11. Ralawise-specific settings:
    
1. in System / Configuration / Auto Product Import / Ralawise,
   
1. Username and password - leave blank to use the default, but you can override this for a specific website if you have website-specific pricing
   
2. Tax classes - you will need to set up tax classes for standard VAT and Zero VAT, and select them here.
   
2. browse to the URL /magmi/web/magmi.php on the site
   
1. if it warns you that /magmi/state is not writeable, create that folder
   
2. next, fix any PHP warnings that show up - eg. permissions issues or config problems
   
3. next, in the top (‘Global’) section of this file, enter your database details and click Save
   
4. then in the next section, from the ‘profile’ dropdown, choose ‘ralawise’
   
5. add a .htaccess file in /magmi with the lines:
   
1. Order Deny,Allow
   
2. Deny from all
   
6. now the products will be able to import
   
3. if not using the Ralawise API, copy the files “catalog.xml” and “prices.csv” from the var/import/ralawise/ on an existing Ralawise import site, to the same place on your new installation.
   
4. if you are using the Ralawise API, edit the file app/code/local/Gateway3D/AutoImport/Model/Datasource/Ralawise.php and change the two lines near the top:
   
1. protected $ftpIsEnabled = false;    ←-- change ‘false’ to ‘true’
   
2. protected $apiIsEnabled = false;   ←-- change ‘false’ to ‘true’
   
12. It should start importing products OVERNIGHT.  If it does not,
    
1. check the cron job has been set up
   
2. enable logging, under System / Configuration / Developer / Log Settings
   
3. then check var/log/system.log - if you see activity in there overnight relating to the import, it is working.
   

New Supplier Questionnaire

If you want to add a new source of product and category data, a technical contact for that company will need to complete the following questionnaire in order to price up the work:

1. What technical protocol will we use to access your product data?
   

• SOAP
  
• Other form of XML request
  
• HTTP request
  
• FTP
  

1. Is it possible to request a small set of products (100-300) at a time in a clearly defined way?  (for example, a whole category, or using a specific search query, but not just “products 100-199 out of 50000”)
   

• Yes, by category
  
• Yes, using a search query
  
• Yes, but only by offset (eg. products 100-199 of 50000)
  
• No, all product data must be received in one file or request
  

1. How will we access product images?
   

• No images available
  
• Full HTTP URL is in the data
  
• Filename or Partial URL is in the data - please describe separately how to create the full HTTP URL using that.
  
• Full image data is embedded in the product data
  
• FTP access using filename in the product data - please provide connection details and how to find the image file in your folder structure
  

1. How will we access category data?
   

• No category data available
  
• Category data is provided separately as a one-off email or download
  
• Same protocol as used for accessing product data (in question 1)
  

1. Are the category names provided suitable for display on a website?
   

• None provided
  
• Yes
  
• No - please explain how the names can be made suitable
  

1. Is the category data provided in a hierarchy?
   

• Yes
  
• No
  

1. Do the categories have images?
   

• Full HTTP URL is in the data
  
• Filename or Partial URL is in the data - please describe separately how to create the full HTTP URL using that.
  
• No
  

1. If access is via SOAP, please specify the following:
   

• Service URL:
  
• WSDL URL:  (normally the same as above, with “?wsdl” added to the end)
  
• Authentication details (eg. username/password)
  

1. How often do you add new categories?
   

• Weekly
  
• Monthly
  
• Annually
  
• For special occasions (eg. Christmas, Summer)
  
• Never
  

1. How often do you delete categories?
   

• Weekly
  
• Monthly
  
• Annually
  
• Never
  

1. How often do you add new products?
   

• Weekly
  
• Monthly
  
• Annually
  
• For special occasions (eg. Christmas, Summer)
  
• Never
  

1. How often do you delete products?
   

• Weekly
  
• Monthly
  
• Annually
  
• On special request
  
• Never
  

1. How often do you need ALL product and category data refreshed on the websites?
   

• minutes
  
• hours
  
• 2 days
  
• 5 days
  
• 1-2 weeks
  

1. Do you need stock and price data to be retrieved live (eg. up to 5 minutes old)?
   

• Yes, stock only
  
• Yes, stock and prices
  
• Yes, prices only
  
• No
  

1. Do you need to be able to manually trigger a category or product to be updated?
   

• Yes
  
• No
  

Timing

In general, the import routine checks each active data source every few minutes to see if it has any work to do.  That work is specific to each data source but consists of these steps:

• download category data
  
• download product data
  
• update Magento categories from the downloaded data
  
• update Magento products from the downloaded data
  

Ralawise

The Ralawise import does each of the above steps as follows:

• download category data: if any categories are older than 18 hours, it will:
  
• re-download all category data
  
• download the brands as categories under a top level “Brands” category
  
• if some categories were missing, mark them as deleted
  
• update Magento categories:
  
• if the user has disabled category import in settings, don’t do this
  
• if it’s a new category, set it to be visible, and to be an “anchor” category
  
• update the name if the config setting allows it
  
• update the image if the config setting allows it
  
• move it if Ralawise have moved it within the category tree
  
• download product data: find a category which has been recently imported and we have not yet downloaded products for, then:
  
• download all the products in that category
  
• if we’ve downloaded a product before, just add it to the new category
  
• download the image, brand, long description and catalogue page
  
• combine the “features”, “sizeDescription” and “icons” fields to create the “short description” for Magento
  
• import all the child products for each master product, along with their sizes, colours, stock level, prices and images
  
• update Magento products:
  
• updates all the fields that we have data for
  
• NOTE - some fields (brand, related items) are only updated if we have data from Ralawise for them
  
• the configurable product (parent product) has its main price set as the cheapest price from the tier prices
  

Sourcing City

G3D ACP

• every ??? hours, it checks all the categories and all the products in ACP
  
• once they are seen, it then imports them into Magento
  

Developer Notes

Using the Real-Time Stock and Price Update

• in System / Config / Gateway3D / Auto Import / General, set Realtime = yes
  
• (this will ensure the javascript is included, and enable realtime data)
  
• in your product view, ensure that the following HTML elements exist, preferably inside “#super-product-table tr” (a <tr> inside <table id=”super-product-table”>):
  
• Stock amount:
  
• a <span> inside “#super-product-table tr” with class “product-stock”
  
• with id=”product-stock-PRODUCT_ID”
  
• Unit price:
  
• <span class=”regular-price” id=”product-price-PRODUCT_ID”>
  
• Unit price quantity (always 1)
  
• <span class=”tpqty” id=”product-price-qty-PRODUCT_ID”>
  
• First tier price:
  
• <span class=”product-tier1” id=”product-tier1-PRODUCT_ID”>
  
• First tier price quantity
  
• <span class=”tpqty” id=”product-tier1-qty-PRODUCT_ID”>
  
• Second tier price:
  
• <span class=”product-tier2” id=”product-tier2-PRODUCT_ID”>
  
• Second tier price quantity
  
• <span class=”tpqty” id=”product-tier2-qty-PRODUCT_ID”>
  

Using Ajax Loading of Blocks to Speed Up Pages

Use this feature if you want to speed up loading of certain pages by making some parts of your HTML load via ajax after the page has been rendered.  You can load them when the page loads, or on demand.

This example was used in the Ralawise project to load a small selection of rows in a table of sub-products when the user clicked a colour.

(apologies for the layout of these instructions - I am happy to answer questions)

• copy the theme file layout/g3dautoimport.xml into your theme folder
  
• the template we want to load our dynamic HTML into is template/catalog/product/view/type/grouped.phtml, and by looking at layout/catalog.xml for that template, we find that the block for that template has name “product.info.grouped”
  
• now, for each bit of dynamic HTML you want to include, do the following:
  
• first choose a name, containing letters, 0-9, dash (-) and dot (.) only - we’ll choose “product.info.grouped.subgroup”
  
• now add a “placeholder” in the HTML, with a unique ID. It can be any element, and the loaded HTML will be placed inside it (innerHTML)
  
•       <tbody id=”product-info-grouped-subgroup-table”/>
  
• now, add code somewhere in our selected template to load the dynamic bit and insert it into the document in the right place.  You need to give it the ID of your HTML element.  It will output javascript with a <script> tag, but if you want to put it in the middle of your own script, call setScriptTag( 0 ) before you call toHtml.  You can also pass it an array of parameters which will be received by your custom template (see below):
  
• <?php
  
•       $ajaxChild = $this->getChild( “product.info.grouped.subgroup” );
  
•       $ajaxChild->setRequestParameters( array( ‘param1’ => 3, ‘param2’ => “hello” );
  
•    Either:
  
•       $ajaxChild->setElementId( “product-info-grouped-subgroup-table” );
  
•    Or:
  
•       $ajaxChild->setCallbackName( “myCallbackFunction” );
  
•   Next, set a custom block name - this is set with the ajax request - see below:
  
•       $ajaxChild->setBlockName( “my-custom-block-name” );
  
•   Then, if you want the script tags disabled, set this to false (default true):
  
•       $ajaxChild->setScriptTagEnabled( 1 );
  
•       echo $ajaxChild->toHtml();
  
• ?>
  
• Alternatively, if you want to write your own version of the ajax request, you can use $ajaxChild->getAjaxUrl() to get the URL for the request.
  
• now create a new template file containing the bit of HTML we want to load dynamically - let’s save it as template/catalog/product/view/type/grouped-ajax.phtml
  
• don’t forget, in this template, the usual variables and methods won’t be available. You’ll need to get data from the request parameters you passed above, eg:
  
• $this->getRequestParameter( ‘param2’ ) // returns “3”
  
• finally, in our copy of layout/g3dautoimport.xml, we will add a special child block to the block that we’re modifying (“product.info.grouped” in this case).  Add the following code inside the <layout> element:
  
• <reference name=”product.info.grouped”>
  
• <block name=”product.info.grouped.ajax” type=”g3dautoimport/ajax” />
  
• </reference>
  
• and to finish off, we need to add your custom template in the layout file - you’ll see an example in there, inside <g3dautoimport_ajax_index>:
  
•    <block name=”my-custom-block-name” type=”core/template” template=”template/catalog/product/view/type/grouped-ajax.phtml” />
  
• Now just clear the cache and you’re done - your dynamic content should load when the loading script runs.  By default this will be when the document is loaded.
  

Adding a new Data Source

• Choose a name - upper case first letter only, eg. “Newsource” not “NewSource”
  
• Add a new set of files under Mode/Datasource:
  
• Model/Datasource/Newsource.php
  
• this class should extend _Model_Datasource_Abstract
  
• it should have the following properties:
  
• ….
  
• it may override the ‘cronjob’  function
  
• if not overriding ‘cronjob’, it must implement:
  
• importCategories
  
• updateMagentoCategories
  
• importProducts
  
• updateMagentoProducts
  
• Model/Datasource/Newsource/Product.php
  
• this class should extend _Model_Datasource_Abstract_Product
  
• it must implement updateMagento
  
• Model/Datasource/Newsource/Category.php
  
• this class should extend _Model_Datasource_Abstract_Category
  
• it must implement updateMagento
  
• The module can create and save _Model_Datasource_Abstract_Product and _Category models, which have the following properties.  How you use them is your choice, but the following is recommended:
  
• datasource_name - the short name, lower case, of the data source class
  
• datasource_code - the unique code within the data source for the product or category
  
• deleted - 0 or 1, to record a deleted product or category
  
• processed - 0 or 1, to tell the ‘updateMagentoProducts’ and ‘updateMagentoCategories’ function whether the product has been updated in Magento
  
• status - a text field you can use in any way you wish, eg. to decide whether a product needs categories
  
• serialised - contains a PHP array, accessed via the setSerialisedData and getSerialisedData methods in _Model_Datasource_Abstract_Product and _Category
  
• Use the existing datasource modules as examples!