Migration of products hiding base images

I am noting that after migrating the products over it seems like it hides some of the product images and I end up having to unhide the images manually.  Is there anyway to fix so that the images stay unhidden?  Is there somewhere it script so I can prevent this?  Alternatively, could I set somewhere in the table to allow all images to display after the migration?
Second question is the module is only migrating 70% of the sales orders over.  Is this because there is some issue with the orders data?  I’m not getting an error message but just a stalled ‘migrating..’ message. 
 

  1. Profile photo of ftoobiftoobi 790.00 $tone March 7, 2020
    That seemed to work but now the order seems off. Even when an image is marked as the 'Base' image in admin, it will always use the image that is in first position in the 'Images and Videos' tab for that product, even if it is not marked as the 'Base' image to be shown in the product detail page. I end up having drag and drop the image that is assigned the 'Base' image to the first position and then flushing cache in order for it to show as the default image on the frontend.

94 answers

Profile photo of Mall Staff 184060.00 $tone February 26, 2020
Public

Hi there,

Alternatively, could I set somewhere in the table to allow all images to display after the migration?

You can run the following SQL query in your M2 database:

Update catalog_product_entity_media_gallery Set disabled = 0;
Update catalog_product_entity_media_gallery_value Set disabled = 0;

Second question is the module is only migrating 70% of the sales orders over.  Is this because there is some issue with the orders data?  I’m not getting an error message but just a stalled ‘migrating..’ message. 

Please continue with the data migration in step #7 by running the CLI command:
php -f bin/ubdatamigration run --step=7

and let me know how it goes.
 
Regards,
Mall.

#1
Profile photo of Mall Staff 184060.00 $tone March 7, 2020
Public

Hi there,

Status: Fail
Message: 

Can you provide me more information about that issue? Alternatively, please provide me information about your instance:

  • Admin credentials of your M2 instance
  • SSH credentials of your M2 instance
  • Let me know the path to your M2 folder

I will help to check further and get back to you then.
 
PS. Please mark your reply private or simply switch this ticket to the private mode, then it’s safe to share your site info here, only you and our technical team can access. 
 
Regards,
Mall.

#3
Profile photo of Mall Staff 184060.00 $tone March 12, 2020
Public

Hi there,
I checked your instance: https://prnt.sc/rf2g00 and saw the following message when trying to run the data migration in step #7:

[joomjiretail@ip-172-31-25-52 public_html]$ php -f bin/ubdatamigration run --step=7
[Processing][run] in step #7:
Status: fail
Message: Reminder! Before migrating data in the step #7, you have to complete migration in the step #5

That was because you had not run the data migration in steps #5, #6 of our migration tool yet.
So, please run the data in step #5,#6 first. And then, you can run data migration in step #7.
php -f bin/ubdatamigration run --step=5
php -f bin/ubdatamigration run --step=6
php -f bin/ubdatamigration run --step=7

Regards,
Mall.

#13
Profile photo of Mall Staff 184060.00 $tone March 12, 2020
Public

Hi there,

Status: fail
Message: CDbCommand failed to execute the SQL statement: SQLSTATE[23000]: Integr ity constraint violation: 1062 Duplicate entry ‘138181’ for key ‘PRIMARY’. The S QL statement executed was: INSERT INTO `sales_order_item` (`order_id`, `weight`, `no_discount`,

That occurred because you checked the ‘KEEP ORIGINAL IDs’ option in step #7 of our migration tool while you had sample/testing data in the sales data section in M2 before you do data migration.

To continue with the data migration in step #7 with the current settings, you have to delete all sample/testing sales data in M2 first.

You can run the following SQL queries in your M2 database to delete all sample/testing sales data:

Delete From sales_order Where entity_id NOT IN (Select m2_id From ub_migrate_map_step_7_order where entity_name = 'sales_flat_order');
Delete From sales_order_item Where item_id NOT IN (Select m2_id From ub_migrate_map_step_7_order_item where entity_name = 'sales_flat_order_item');
Delete From sales_order_address Where entity_id NOT IN (Select m2_id From ub_migrate_map_step_7_order_address where entity_name = 'sales_flat_order_address');
Delete From quote Where entity_id NOT IN (Select m2_id From ub_migrate_map_step_7_quote where entity_name = 'sales_flat_quote' OR entity_name = 'sales_flat_quote_active');
Delete From quote_item Where item_id NOT IN (Select m2_id From ub_migrate_map_step_7_quote_item where entity_name = 'sales_flat_quote_item');
Delete From quote_address Where address_id NOT IN (Select m2_id From ub_migrate_map_step_7_quote_address where entity_name = 'sales_flat_quote_address');

Once done, you can continue with the data migration in step #7 by running the CLI command:
php -f bin/ubdatamigration run --step=7

Regards,
Mall.

#15
Profile photo of Mall Staff 184060.00 $tone March 13, 2020
Public

Hi there,

I’ve done the following steps outlined in your previous post (received an error when I tried to delete ‘sales_flat_quote’ table because it didn’t exist in the database)

Please let me know which post you referred to? Please note that there is no SQL query to delete data in the table named ‘sales_flat_quote’ in my reply #15. Please check this again.

Message: CDbCommand failed to execute the SQL statement: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘66674’ for key ‘PRIMARY’. The SQL statement executed was: INSERT INTO `quote_payment` (`quote_id`, `cc_exp_year`, `cc_ss_start_month`,

To solve that issue, you need to delete sample/testing data in the quote_payment table in M2. You can run the following SQL query in your M2 database:

Delete From quote_payment Where payment_id = 66674;

+ Once done, you can continue with the data migration in step #7.

Regards,
Mall.
 

#17
Profile photo of ftoobi 790.00 $tone March 16, 2020
Public

I’m getting this error now:
CDbCommand failed to execute the SQL statement: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘138181’ for key ‘PRIMARY’

#18
Profile photo of Mall Staff 184060.00 $tone March 16, 2020
Public

Hi there,

CDbCommand failed to execute the SQL statement: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘138181’ for key ‘PRIMARY’

Please provide me more details about that error message, I will check and get back to you then.

Regards,
Mall.

#19
Profile photo of ftoobi 790.00 $tone March 16, 2020
Public

That was all it showed.  I reset and then remigrated.  It seemed to be migrated (although at a very slow pace) and about 30 min later it stopped migrating.

#20
Profile photo of Mall Staff 184060.00 $tone March 17, 2020
Public

Hi there,

That was all it showed.  I reset and then remigrated.  It seemed to be migrated (although at a very slow pace) and about 30 min later it stopped migrating.

I checked again in your M2 instance and I saw the issue you mentioned happened in step #7 of our modue:
https://prnt.sc/rhkaem, https://prnt.sc/rhkahz

CDbCommand failed to execute the SQL statement: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘138181’ for key ‘PRIMARY’

I have checked further via SSH and I saw the fully of that error message as following:

[Processing][run] in step #7: ..
Status: fail
Message: CDbCommand failed to execute the SQL statement: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘138181’ for key ‘PRIMARY’. The SQL statement executed was: INSERT INTO `sales_order_item` (`….

To solve that issue you can do steps as following:

+ Run following SQL in your M2 database:

Delete From sales_order_item Where item_id = 138181;

+ Once done, you can continue with data migration in step #7:
php -f bin/ubdatamigration run --step=7;
And tell me know how it goes.
Regards,
Mall.
 

#21
Profile photo of ftoobi 790.00 $tone March 19, 2020
Public

WE ARE REALLY DISAPPOINTED WITH THIS MODULE SO FAR.
You need to spend the time and then do the FULL migration yourself and then need to give us a code that we can run again on a clean database to make sure we have a code that works.
Again, you need to do the FULL conversion. We do not have time to go back and forth on this. You need to be professional and do this and do it correctly.
Also, why are there constraint issues?!!!! Are you not creating data in order? There should not be a case where you remove a constraint in this code unless the code is fundamentally not correct and you are not importing data in the correct order.
Fix this immediately. Do all conversion yourself. DO NOT KEEP TELLING US TO SO STUFF.
And I need to know when this will be completed.
 

#23
Profile photo of ubdev Staff 98150.00 $tone March 20, 2020
Public

Hi there,
 
We’d like to clarify a little bit further below:
 
Through this ticket, you did submit the following issues:

#1: could I set somewhere in the table to allow all images to display after the migration?

We did provide instruction in our reply #1.

#2: Second question is the module is only migrating 70% of the sales orders over. Is this because there is some issue with the orders data? I’m not getting an error message but just a stalled ‘migrating..’ message.
[joomjiretail@ip-172-31-25-52 public_html]$ php -f bin/ubdatamigration run --step=7
[Processing][run] in step #7:
Status: fail

This related to your migration process at your end only. You proceeded with Step 7 while not running the data migration in steps #5, #6 of our migration tool yet.

#3: Status: fail
Message: CDbCommand failed to execute the SQL statement: SQLSTATE[23000]: Integr ity constraint violation: 1062 Duplicate entry ‘138181’ for key ‘PRIMARY’. The S QL statement executed was: INSERT INTO `sales_order_item` (`order_id`, `weight`, `no_discount`,
#4. [Processing][run] in step #7: ..
Status: fail
Message: CDbCommand failed to execute the SQL statement: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘138181’ for key ‘PRIMARY’. The SQL statement executed was: INSERT INTO `sales_order_item` (`….

Both issues #3 & 4 occurred because you checked the ‘KEEP ORIGINAL IDs’ option in step #7 of our migration tool while you had sample/testing data in the sales data section in M2 before/ after you run data migration in step 7.
 
Regarding the Keep Original IDs setting, we did specify a detailed note right within the configuration Dashboard (like this screenshot). It notified you that it’s mandatory that your Magento 2 must be a fresh installation when selecting this option.
 
Data migration is a complex process that associates with various factors, including your database itself, so please work closely with us on each single issue.
 
Sincerely,
Ubertheme team

#24
Profile photo of Mall Staff 184060.00 $tone March 20, 2020
Public

Hi there,

 I reset the order migration but still getting those constraint errors.

That still came from the fact that you checked the ‘KEEP ORIGINAL IDs’ setting in step #7 and after you reset step #7, the testing data in the sales data section still remains (The Reset function of our module only covers only the migrated items). 

To continue with the data migration in step #7 with the ‘KEEP ORIGINAL IDs’ setting, deleting all testing data in the sales data section is still needed first.

Regards,
Mall.

#26
Profile photo of Mall Staff 184060.00 $tone March 20, 2020
Public

Hi there,

I have done this already.

I helped to check your M2 database further and I saw that there are still existing sales data (sample or testing data) added in M2 which were not migrated by our migration tool:
You can learn more about that in this screenshot: http://prntscr.com/rjgfdw

As explained, you have to delete all that data before you can do data migration in step #7 with the ‘KEEP ORIGINAL IDs’ enabled.

So, please do the following steps at your end:

+ Run the SQL queries below in your M2 database to delete all testing/sample data in the sales data section:

Delete From salesrule Where rule_id NOT IN (Select m2_id From ub_migrate_map_step_7 where entity_name = 'salesrule');
Delete From quote Where entity_id NOT IN (Select m2_id From ub_migrate_map_step_7_quote where entity_name = 'sales_flat_quote' OR entity_name = 'sales_flat_quote_active');
Delete From quote_item Where item_id NOT IN (Select m2_id From ub_migrate_map_step_7_quote_item where entity_name = 'sales_flat_quote_item');
Delete From quote_address Where address_id NOT IN (Select m2_id From ub_migrate_map_step_7_quote_address where entity_name = 'sales_flat_quote_address');
Delete From sales_order Where entity_id NOT IN (Select m2_id From ub_migrate_map_step_7_order where entity_name = 'sales_flat_order');
Delete From sales_order_item Where item_id NOT IN (Select m2_id From ub_migrate_map_step_7_order_item where entity_name = 'sales_flat_order_item');
Delete From sales_order_address Where entity_id NOT IN (Select m2_id From ub_migrate_map_step_7_order_address where entity_name = 'sales_flat_order_address');

+ Once done, you can continue with the data migration in step #7:
php -f bin/ubdatamigration run --step=7;

Regards,
Mall.

#28
Profile photo of ftoobi 790.00 $tone March 23, 2020
Public

It looks like there are still 6,045 sales records that need to be imported.  It is the record with blank value causing it to halt the migration.  Please let me know how to locate that record so I can fix and complete the migration.
See below screenshots:
https://prnt.sc/rlj2p7 (order records imported so far)
https://prnt.sc/rlj3jm (total order records)

#30
Profile photo of Mall Staff 184060.00 $tone March 24, 2020
Public

Hi there,

https://prnt.sc/rliy9o (error when remigrate sales orders)
 It is the record with blank value causing it to halt the migration.

That issue was because your M1 database had at least one data record which had the bad data in the ‘order_status’ field in the ‘sales_order_aggregated_created’ table.
You can run the following SQL in your M1 database to seek for those data records at your end:

Select * From sales_order_aggregated_created Where order_status = '' OR order_status IS NULL;

+ And then, you have re-fill needed value of the ‘order_status’ field on all found data records. Or you can delete these data records.
 
+ Once done you can continue with the data migration in step #7:
php -f bin/ubdatamigration run --step=7;
 
Regards,
Mall.

#31
Profile photo of ftoobi 790.00 $tone March 26, 2020
Public

That worked and was able to migrate all sales records but I am still seeing that the order page shows 52186 records  and on Step 7 of the  migration page it shows  the number of Sales Orders at 58231.  Shouldn’t both these numbers be matching?
 
 

#32
Profile photo of Mall Staff 184060.00 $tone March 26, 2020
Public

Hi there,
I have checked your instance http://joomjiretail.com/admin_qadb2z further. And I saw that:

  • Your M1 currently has total 59853 sales orders: http://prntscr.com/rmxr0g
  • Our tool helped to migrate total 58231 sales orders: http://prntscr.com/rmxrmu

Meanwhile I saw the ‘migrating’ status for customers migration (step #6): https://prnt.sc/rmxssc

So, please continue to finish data migration in step #6. And then continue to run the delta migration in step #7.

Once done, check the issue you mentioned once again.
 
Regards,
Mall.

#33
Profile photo of ftoobi 790.00 $tone March 26, 2020
Public

I  just wanted to make sure that all orders have been imported.  Also, I followed instructions on running the delta migration but when I changed the database settings in step #1 to point to the remote production database the ‘Delta’ buttons in step #7 turned to ‘Run’ buttons and I ended up having to start the migration over by clicking the ‘Run’ buttons.  My assumption is that after changing the database to point to the remote version, the buttons would stay as ‘Delta’ and I would just click each one to only migrate the new data from the production server.  Please advise.

#34
Profile photo of Mall Staff 184060.00 $tone March 27, 2020
Public

Hi there,

Now I need to run the delta migration. Please let me know best way to do that. 

Please follow our instruction about the delta migration in the Readme manual (included in your download package). 
In case you still have any additional question, please let us know here. 

Regards,
Mall.

#36
Profile photo of ftoobi 790.00 $tone March 30, 2020
Public

I was able to delta migrate: 1,2,4,5
when I tried to delta migrate attributes (step 3), it looked to be migratingin from the command line but stalled after 3 minutes and threw error message in ssh: 
PHP Error[2]: PDOStatement::execute(): MySQL server has gone away
in file /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/db/CDbCommand.php at line 508
#0 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/db/CDbCommand.php(508): PDOStatement->execute()
#1 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/db/CDbCommand.php(415): CDbCommand->queryInternal()
#2 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/db/ar/CActiveRecord.php(1359): CDbCommand->queryRow()
#3 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/db/ar/CActiveRecord.php(1464): Mage1AttributeSet->query()
#4 /home/joomjiretail/public_html/pub/ub-tool/protected/models/UBMigrate.php(1366): Mage1AttributeSet->find()
#5 /home/joomjiretail/public_html/pub/ub-tool/protected/controllers/Step3Controller.php(476): getM1EntityTypeCode()
#6 /home/joomjiretail/public_html/pub/ub-tool/protected/controllers/Step3Controller.php(201): Step3Controller->_migrateAttributeGroups()
#7 /home/joomjiretail/public_html/pub/ub-tool/protected/commands/RunCommand.php(74): Step3Controller->actionRun()
#8 /home/joomjiretail/public_html/pub/ub-tool/protected/commands/RunCommand.php(33): RunCommand->_migrateData()
#9 unknown(0): RunCommand->actionIndex()
#10 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/console/CConsoleCommand.php(172): ReflectionMethod->invokeArgs()
#11 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/console/CConsoleCommandRunner.php(71): RunCommand->run()
#12 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/console/CConsoleApplication.php(92): CConsoleCommandRunner->run()
#13 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/base/CApplication.php(185): CConsoleApplication->processRequest()
#14 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/yiic.php(33): CConsoleApplication->run()
#15 /home/joomjiretail/public_html/pub/ub-tool/protected/ubdatamigration_cli.php(5): require_once()
#16 /home/joomjiretail/public_html/bin/ubdatamigration(4): require_once()
[joomjiretail@ip-172-31-25-52 public_html]$
 
When I tried to delta migrate step 6 (customers) from magento admin, I received the following error message:
CDbCommand failed to execute the SQL statement: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘14088’ for key ‘PRIMARY’
 
Note, all buttons are set to Delta.

#37
Profile photo of Mall Staff 184060.00 $tone March 31, 2020
Public

Hi there,

but stalled after 3 minutes and threw error message in ssh: 
PHP Error[2]: PDOStatement::execute(): MySQL server has gone away

Mage1AttributeSet->query()

I saw that this referred to a MySQL server issue only. You have lost the connection to MySQL server of your M1 database at that time.

Was your M1 database on the same Server with M2 database?

When I tried to delta migrate step 6 (customers) from magento admin, I received the following error message:
CDbCommand failed to execute the SQL statement: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘14088’ for key ‘PRIMARY’

That occurred because you checked the ‘KEEP ORIGINAL IDs’ setting in step #6 while you added more customer’s data in M2 after the first migration. So, please continue with the data migration in step #6 by running the CLI command:
php -f bin/ubdatamigration run --step=6
Then, provide me more details of error message you saw. Based on that, I will suggest a workaround to solve that issue.

https://prnt.sc/rptnvf

That screenshot indicates that you first got a connection issue with your M1 database (PDOStatement::execute(): MySQL server has gone away). Please consider solving the  connection issue with your M1 database first.

Regards,
Mall.
 

#39
Profile photo of ftoobi 790.00 $tone March 31, 2020
Public

Please check the screenshot where it shows all items have been migrated (https://prnt.sc/rpyd7b.) The issue is when I switch to the remote database it will stall for step #7, throw error for step #6 and show sql has gone away for step #3.  It does this every time I run the delta migration.

#40
Profile photo of Mall Staff 184060.00 $tone March 31, 2020
Public

Hi there,

Please check the screenshot where it shows all items have been migrated (https://prnt.sc/rpyd7b.)

Yes, I see you have proceeded data migration in all steps with a backup of your M1 database.

The issue is when I switch to the remote database it will stall for step #7, throw error for step #6 and show sql has gone away for step #3.  It does this every time I run the delta migration.

So, you should contact the Hosting provider or System admin of your M1 server to solve the issue “MySQL has gone away” for step #3 first.

And then, after you can finish the delta migration in step #3, please continue with the data migration in step #6 by running the CLI command: php -f bin/ubdatamigration run --step=6
 
Then, provide me more details of error message you saw. Based on that, I will suggest a workaround to solve that issue.
 
Regards,
Mall.

#41
Profile photo of ftoobi 790.00 $tone April 4, 2020
Public

Please provide an answer regarding:
 
When I run the delta migration for step 6 it shows that the migration has been completed but I see that no migration actually took place when I checked the customers grid page in admin. (see https://prnt.sc/rsqsfh that shows after /usr/local/bin/php -f bin/ubdatamigration run --step=6 --mode=update is ran and https://prnt.sc/rsquh2 which shows no new customer records came through from the delta migration). If delta migration were working it would have showed customer records from today (4/3/20). Instead, the latest customer record is from early February when the full migration was done. Please fix asap as we have to launch soon.

Thanks

#43
Profile photo of ftoobi 790.00 $tone April 5, 2020
Public

I was able to get all of the customers to migrate by removing all of the customer records and remigrating from the remote database instead of the local one.

#44
Profile photo of Mall Staff 184060.00 $tone April 6, 2020
Public

Hi there,

When I run the delta migration for step 6 it shows that the migration has been completed but I see that no migration actually took place when I checked the customers grid page in admin. (see https://prnt.sc/rsqsfh that shows after /usr/local/bin/php -f bin/ubdatamigration run --step=6 --mode=update is ran and https://prnt.sc/rsquh2 which shows no new customer records came through from the delta migration)

It seems that you haven’t reindexed data successfully yet, after you finished delta migration in step #6.

I was able to get all of the customers to migrate by removing all of the customer records and remigrating from the remote database instead of the local one.

So, please continue with your current instance and let me know if you need further assistance.

Regards,
Mall.

#45
Profile photo of ftoobi 790.00 $tone April 7, 2020
Public

I have been able to do a full migration and the delta migration is working.  The issue I am facing now is each time I do a delta migration for the products, it gets sent to the ‘default category migrated’ folder and I end up having to move those over to the ‘default category’ folder.  Is there a simpler way to go about this so I don’t have to keep transferring over the category folder from ‘default category migrated’ to ‘default category’ parent folder each time I do a delta migration for products?  Also, I’m also noticing a few products migrated over without the images.
Thanks

#46
Profile photo of ubdev Staff 98150.00 $tone April 8, 2020
Public

Hi there, 

The issue I am facing now is each time I do a delta migration for the products, it gets sent to the ‘default category migrated’ folder and I end up having to move those over to the ‘default category’ folder.

You shouldn’t do that. You should set the Migrated Default Category as the Root category of the current default Store of the default website in your M2. You can change this setting in the Stores managements in your M2 backend.

Also, I’m also noticing a few products migrated over without the images.

Please make sure you already moved/copied all media images from Magento 1 as per our Readme manual. In addition, any media images exluded in M1 will be marked as hidden images after being migrated to M2.

Regards,
Ubertheme team
 

#47
Profile photo of ftoobi 790.00 $tone April 8, 2020
Public

Ok, thanks.  We are also noticing that some of the Magento 1 configurable products that are being migrated over have special price and regular price set instead of on the simple products.  When those configurable products get transferred over to Magento 2 the special price is not used because it was added on the configurable product in Magento 1 and it appears Magento 2 uses the simple product prices and special prices instead of that set on the configurable product.  Is there a way around this as it seems the configurable products that had regular and special price assigned in Magento 1 are not being recognized as being on sale when migrated over to Magento 2.
Thanks

#48
Profile photo of ftoobi 790.00 $tone April 8, 2020
Public

Also,
I am noticing when I run the Delta migration on products that it only migrates new products that were added in Magento 1.  What about products that were already migrated but recently changed after the delta migration (ie: inventory, product name, special price added to simple product, etc.?) How do I bring the changed data for existing migrated products over to magento 2?  Do I have to do another full migration for the products? 

#49
Profile photo of Mall Staff 184060.00 $tone April 8, 2020
Public

Hi there,

We are also noticing that some of the Magento 1 configurable products that are being migrated over have special price and regular price set instead of on the simple products.  When those configurable products get transferred over to Magento 2 the special price is not used because it was added on the configurable product in Magento 1 and it appears Magento 2 uses the simple product prices and special prices instead of that set on the configurable product

That is a new working rule of Configurable products in M2. If you add a new Configurable product in M2, you will learn more about that. Also, you can find out more about M2’s Configurable product settings via this guide.

 What about products that were already migrated but recently changed after the delta migration (ie: inventory, product name, special price added to simple product, etc.?) How do I bring the changed data for existing migrated products over to magento 2?

To delta update migrated products, you have to run delta migration in step #5 with the ‘update’ mode by running the CLI command:
php -f bin/ubdatamigration run --step=5 --mode=update;

Regards,
Mall.

#50
Profile photo of ubdev Staff 98150.00 $tone April 8, 2020
Public

Hi there,
It’s a pity that we do not have any suggestion in place at the moment. Anyway, it would be best to follow the new working rule of Magento 2 as mentioned in our reply #50 above. 
Regards,
Ubertheme team

#52
Profile photo of Mall Staff 184060.00 $tone April 9, 2020
Public

Hi there,

and noted that the category pages show a 404 page (see http://joomjiretail.com/americana.html) on frontend even after reindexing and clearing cache.  Please advise.

The not found issue occurred because there is no URL Rewrite for that Category’s Request path (americana.html) in your M2:
https://prnt.sc/rvtm6i
https://prnt.sc/rvtmi3

As you see, there is a Category’s URL rewrite with the Request Path ‘camouflage.html’
And it was working fine in your frontend: http://joomjiretail.com/camouflage.html

You can try with the following steps so that M2 can help regenerate URL rewrite for categories and associated products:
1 -- In categories management, create a new root category named ‘Temp Root Category’
2 -- Open Stores management, edit the default Store of the default Website and set the ‘Root Category’ form field to the newly added root category ‘Temp Root Category’ and click Save button.
3 -- Once done, open Stores management once again, edit the default Store of the default Website and set the Root Category form field to the main root category which you want to publish in the storefront again and click Save button.

Magento will regenerate categories and associated products URLs rewrite in this step. It might take sometime to complete this task (It depends on the volume of categories and associated products)

Once done, re-index the data and clean the Magento cache.
 
And tell me know how it goes then.
 
Regards,
Mall.

#54
Profile photo of ftoobi 790.00 $tone April 9, 2020
Public

I followed your steps but  I see that the url rewrites is stuck at 615 records even after reindexing and flushing cache.   How come it worked for camouflage and Velcro categories but not the other categories?  If getting the category pages to show on the front end means resetting and starting the product migration step again then I can do that but this needs to be resolved as we are trying to launch soon.  
Thanks

#55
Profile photo of Mall Staff 184060.00 $tone April 10, 2020
Public

Hi there,

If getting the category pages to show on the front end means resetting and starting the product migration step again then I can do that but this needs to be resolved as we are trying to launch soon.  

No, that is not needed.

I followed your steps but  I see that the url rewrites is stuck at 615 records

Yes, I have tested those steps at my end and saw that issue too. This is strange.
 
I have enabled the ‘developer’ mode in your instance to debug further and noticed the following error in the frontend when accessing this URL: http://joomjiretail.com/camouflage.html

1 exception(s):
Exception #0 (Magento\Framework\Exception\LocalizedException): The XML in file “/home/joomjiretail/public_html/app/design/frontend/Infortis/ultimo/etc/view.xml” is invalid:
Element ‘aspect_ratio’: This element is not expected. Expected is ( background ).
Line: 21

As can be seen, that issue related to a bad XML declaration of the third-party module named ‘Infortis/ultimo’. Thus, you should consider getting this issue fixed first.

Also, I saw there is another third-party module named ‘Infortis_UltraMegamenu’ which may affect Magento’s generating category’s URL rewrite function. And it might be the root cause of the issue you mentioned. Could you disable all third-party modules in your site to test the data migrated by our migration tool first?

Regards,
Mall.

#56
Profile photo of ftoobi 790.00 $tone April 10, 2020
Public

Hi, I removed the aspect_ratio but now I see the product images are not a perfect square like they are on the production site.  Is there another way to get the images to show the dimensions that have the same height and width with no whitespace background like before? It seemed that :
<transparency>false</transparency>
<aspect_ratio>false</aspect_ratio>
<frame>false</frame>
worked before but all of a sudden it is throwing an error.
 
Also, I disabled the infortis megamenu but and I see the remaining products are showing but at the cost of using the default magento category url (ie: http://joomjiretail.com/catalog/category/view/s/denim/id/171/) instead of the category slug (ie: joomjiretail.com/denim.html) Is it like this because url rewrite is not working?  The urls need to be same as the production site for seo purposes.  Once that issue is resolved I will reenable infortis megamenu.  

#57
Profile photo of Mall Staff 184060.00 $tone April 10, 2020
Public

Hi there,

Is there another way to get the images to show the dimensions that have the same height and width with no whitespace background like before?

You should contact your third-party theme provider to get better help on that issue.

http://joomjiretail.com/catalog/category/view/s/denim/id/171/) instead of the category slug (ie: joomjiretail.com/denim.html)

That was because that the category (id 171) still had missing URL Rewrite. Please try to re-peform steps as per my reply #54 so that M2 can re-generate URL rewrites.
 
Regards,
Mall.

#58
Profile photo of ftoobi 790.00 $tone April 13, 2020
Public

I forgot to mention,
when running the delta migration on products, is there a way to bypass updating certain attributes on existing products such as  the ‘special price’ attribute?  I do not want the delta migration to overwrite the ‘special prices’ values. All I need the delta migration for at this point is to update inventory values.
If this is not possible, can the module be modified by commenting out the section that delta migrates over special prices?
Thanks

#60
Profile photo of Mall Staff 184060.00 $tone April 13, 2020
Public

Hi there,

 but now I am seeing dropdown list for sizes is out of order (see https://prnt.sc/ry8glt)

That doesn’t relate to our migration tool. Instead, it refers to Magento’s function or third-party theme you are using.

If this is not possible, can the module be modified by commenting out the section that delta migrates over special prices?

Yes, you can apply with these steps:
+ Open the PHP file at pub/ub-tool/protected/controllers/Step5Controller.php
and find the lines:

if ($productTypeId == 'configurable') {
$ignoreAttributeIds = array_merge($ignoreAttributeIds, array (
UBMigrate::getMage1AttributeId('price', $entityTypeId),
UBMigrate::getMage1AttributeId('special_price', $entityTypeId),
UBMigrate::getMage1AttributeId('special_from_date', $entityTypeId),
UBMigrate::getMage1AttributeId('special_to_date', $entityTypeId),
));
}

and replace it with:

if ($productTypeId == 'configurable') {
$ignoreAttributeIds = array_merge($ignoreAttributeIds, array (
UBMigrate::getMage1AttributeId('price', $entityTypeId),
UBMigrate::getMage1AttributeId('special_price', $entityTypeId),
UBMigrate::getMage1AttributeId('special_from_date', $entityTypeId),
UBMigrate::getMage1AttributeId('special_to_date', $entityTypeId),
));
}
if ($this->runMode == UBMigrate::RUN_MODE_DELTA) {
$ignoreAttributeIds = array_merge($ignoreAttributeIds, array(
UBMigrate::getMage1AttributeId('special_price', $entityTypeId),
UBMigrate::getMage1AttributeId('special_from_date', $entityTypeId),
UBMigrate::getMage1AttributeId('special_to_date', $entityTypeId),
));
}

Once done, you can ignore the product’s special price in the delta migration with the ‘update’ mode in step #5.

Regards,
Mall.
 

#61
Profile photo of Mall Staff 184060.00 $tone April 20, 2020
Public

Hi there,
I am sorry for replying to you a bit late as we’ve just been back from the weekend. 

How about ignoring media galleries on next delta migration?

Product’s media images are required and migrated by default in step #5 to ensure the product’s data relationship.
 
If you still need to ignore product’s media images in the next delta migration in step #5, you can apply these steps:

+ Open the PHP file at pub/ub-tool/protected/controllers/Step5Controller.php
and find the code line:

$this->_migrateCatalogProductGallery($product->entity_id, $product2->entity_id, $mappingStores, $mappingAttributes, $keepOriginalId);

and replace it with:

//$this->_migrateCatalogProductGallery($product->entity_id, $product2->entity_id, $mappingStores, $mappingAttributes, $keepOriginalId);

+ Once done, you can do data migration in step #5 while ignoring the product’s media galleries.

Regards.
Mall.
 

#63
Profile photo of Mall Staff 184060.00 $tone April 24, 2020
Public

Hi there,

Another issue I noticed is that I see ‘out of stock’ and ‘product qty not available’ if the quantity is 2 or below 2.  

Please provide me more details about that issue. 

I’m noticing that during delta migrations where an order took place on the production site, it will not be reflected on the inventory migrated over.  For example, if a sale was made on a product, the inventory would be reduced by 1, but when the latest inventory is sent over for that product in the delta migration, it will not reflect the current inventory as if the sale of that product never happened.

The delta function in step #5 of our module covers both newly added products and migrated Products which have changes in M1 after the first migration (The catalog_product_entity.updated_at with changed datatime value)

The context of place order on migrated product in M1 site you mentioned wasn’t updated with the value of catalog_product_entity.updated_at field after M1 reduced the quantity of this product in M1 database. Thus the delta function in step #5 of our module couldn’t handle the changes for this case.

To resolve this case, you can manually update the datetime value of the ‘catalog_product_entity.updated_at‘ field in M1 database for the product which had captured the order in M1. And then, you run the delta migration in step #5 with the ‘update’ mode by running the CLI command:
php -f bin/ubdatamigration run --step=5 --mode=update;

Regards,
Mall.
 
 

#65
Profile photo of ubdev Staff 98150.00 $tone April 27, 2020
Public

Hi there,
We’d like to follow up further below: 

why wasn’t it updated? I’m not sure I understand what this means. Is there something I did wrong during the migration? I followed the steps of doing a base migration from M1 database and then followed up with delta migrations from the remote production database. Did I miss a step?

This relates to the Magento’s functions, not the wrong steps of migration at your end.

I am not sure what you mean by manually update the datetime value? Do I have to actually go to phpMyAdmin and update values in the ‘catalog_product_entity.updated_at’ field and change the values for each new order? I also don’t understand why I have to ‘manually’ update when the data migration module should be doing this automatically.

As explained above, the catalog_product_entity.updated_at value is the time base for the Delta functions of our migration tool which allows filtering which products need to be delta-migrated in step #5 (products with changes in M1)
If you can not manually update case by case after having new orders in M1, you can run the following SQL query to delta update all products in your M1 database (NOTE: with this workaround, the delta migration will take a rather long time as the first migration in step #5):
1. Run the following SQL in your M2 database:

Update ub_migrate_map_step_5 Set created_time = '2000-01-01 00:00:00' Where entity_name = 'catalog_product_entity';

2. Run the delta migration in step #5 with the ‘update mode’:
php -f bin/ubdatamigration run --step=5 --mode=update
 
We hope that helps.
Regards,
Ubertheme team
 

#67
Profile photo of Mall Staff 184060.00 $tone April 27, 2020
Public

Hi there,

 I tried running the 
I only found the ub_migrate_map_step_5 table on the target database (joomjire_dgg8) and not the base m1 database (joomjire_dgg11).

Yes, you need to run the SQL query I suggested in M2 database (I have just edited our reply #67)

but it threw an error when I ran the php -f bin/ubdatamigration run --step=5 --mode=update command so I restored the ‘ub_migrate_map_step_5’ table.  

Please redo steps as suggested in our reply #67. And provide me details about the error message you mentioned in step #5. I will help to check further and get back to you then.

Regards,
Mall.
 

#69
Profile photo of ftoobi 790.00 $tone April 27, 2020
Public

I get this message:
Status: fail
Message: The table “{{inventory_source_item}}” for active record class “Mage2InventorySourceItem” cannot be found in the database.
[joomjiretail@ip-172-31-25-52 public_html]$

#70
Profile photo of Mall Staff 184060.00 $tone April 27, 2020
Public

Hi there,

Message: The table “{{inventory_source_item}}” for active record class “Mage2InventorySourceItem” cannot be found in the database.

What is the version of your M2 instance? That issue occurred because you had not installed Magento_Inventory module. Please consider enabling this module in your M2 instance. 
Once done, you can continue with the data migration in step #5 of our module.

Regards,
Mall.

#71
Profile photo of ftoobi 790.00 $tone April 28, 2020
Public

I disabled MSI as it was causing issues with salable stock and actual inventory.  I reenabled MSI, ran the query and the update command and now inventory is up to date. 

#72
Profile photo of ubdev Staff 98150.00 $tone April 28, 2020
Public

Hi there, 
Yes, please continue with the process at your end. If you need our further assistance, please get back back to us here.
Regards,
Ubertheme team

#73
Profile photo of Mall Staff 184060.00 $tone May 4, 2020
Public

Hi there,
I am sorry for replying to you this late as we’ve just been back to work from our weekend. 

I am noticing that there are about half a dozen products that have salable quantity at 0 and quantity at a different number even after reindexing when the delta migration is completed.  How can this be corrected so that salable quantity is the same as regular quantity after delta migrating the products?  

That occurred because you didn’t enable ‘MSI’ in the first migration. Please do additional steps as follows:

+ Run the following SQL query in your M2 database:

Update ub_migrate_map_step_5 Set created_time = '2000-01-01 00:00:00' Where entity_name = 'catalog_product_entity';

+ Once done, you run the delta migration in step #5 with the ‘update’ mode:
php -f bin/ubdatamigration run --step=5 --mode=update;
(Please note this can take a long time because it scans all products)
 
Regards,
Mall.

#75
Profile photo of ftoobi 790.00 $tone May 4, 2020
Public

I already took this step and it worked for the majority of products (salable qty and regular qty were synced).  There were about a dozen products that were out of sync. Is there a way to manually put theses in sync?

#76
Profile photo of Mall Staff 184060.00 $tone May 5, 2020
Public

Hi there,

There were about a dozen products that were out of sync.

That is strange if you already did complete all steps as suggested in my reply #75. Please try to reindex the data and check the issue you mentioned once again.

If you need further help, please provide me information of your instance:

  • URL and Admin credentials of your M2 instance
  • SSH credentials of your M2 instance
  • Let me know the path to your M2 folder
  • All product IDs or Product’s SKUs which had the issue you mentioned (a dozen products that were out of sync)

I will help to  check further and get back to you then.
 
PS. Please make sure you mark your reply private when sharing your site info. Then only you and our technical team can access it. 

Regards,
Mall.

#77
Profile photo of ftoobi 790.00 $tone May 20, 2020
Public

Hi,
 
I reran a delta migration and all migrated up to date with exception of the last 25 customers.  I reindex but still the last 25 customers did not get included in the migration.  I did not receive an error message from the command line so not sure why the migration was not completed for customers.
 

#78
Profile photo of Mall Staff 184060.00 $tone May 20, 2020
Public

Hi there,

 I reindex but still the last 25 customers did not get included in the migration.

Please make sure that you did reindex data successfully after you finished the data migration in step #6.

Also, you can directly check the total of migrated customers in the ‘customer_entity’ table in your M2 database.
 
Regards,
Mall.

#79
Profile photo of ftoobi 790.00 $tone May 20, 2020
Public

I redinedex and confirmed that the discrepancy is 29 customers that did not get imported after viewing the ‘customer_entity’ table.  What else should I do to get customers current?  This is the only issues holding us up from going live.  Should I reset customers step and import the customers again?  Please advise.

#80
Profile photo of Mall Staff 184060.00 $tone May 21, 2020
Public

Hi there,

 Should I reset customers step and import the customers again?  Please advise.

Please try with these steps:
+ Run the following SQL query in your M2 database:

Update ub_migrate_map_step_6 Set created_time = '2000-01-01 00:00:00' Where entity_name = 'customer_entity';

+ Then, you run the delta migration in step #6 with the ‘update’ mode:
php -f bin/ubdatamigration run --step=6 --mode=update;
+ Once done, reindex the data again. And let me know how it goes.

Regards,
Mall.

#81
Profile photo of Mall Staff 184060.00 $tone May 21, 2020
Public

Hi there,

Still short 25 customers after running the sql query and the update command.  I also reindexed.

That is strange. Please provide me information of your instance:

  • URL and Admin credentials of your M2 instance
  • SSH credentials of your M2 instance
  • Let me know the path to your M2 folder
  • phpMyadmin credentials of your M1 and M2 databases.

I will help to check further and get back to you then.
 
PS. Please make sure you mark your reply private or simply switch this ticket to the private mode, then it’s safe to share your site info here. Only you and our technical team can access it. 

Regards,
Mall.
 

#83
Profile photo of Mall Staff 184060.00 $tone May 22, 2020
Public

Hi there,

 Still short 25 customers after running the sql query and the update command.  I also reindexed.

I have checked your site and saw that our tool helped to migrate total 14600 customers: http://prntscr.com/slhxyk
 
And I saw that there are total 14603 customers in the Customer grid view in your M2 backend: https://prnt.sc/slhy4s
 
So, please check the issue you mentioned again.
 
Also, when accessing the settings in step #6 of our module, I got this issue: http://prntscr.com/slhzh3
 
Regards,
Mall.

#85
Profile photo of Mall Staff 184060.00 $tone May 24, 2020
Public

Hi there,

The total customer count is currently at:  14634 on the M1 site (see: https://prnt.sc/sluojj). 

So, please re-perform all steps as suggested in my reply #81 to migrate the missing customers at the current time. Then let me know how it goes.

Regards,
Mall.

#87
Profile photo of Mall Staff 184060.00 $tone May 25, 2020
Public

Hi there,

running the query and update command worked this time.

I am glad to hear that. Please continue the process at your end and let me know if you need further assistance.

Regards,
Mall.

#89
Profile photo of ftoobi 790.00 $tone June 8, 2020
Public

Hi,
I’m having a different issue with the delta step #6 migration.  It is throwing a php error about 20 minutes into migration after running the following commands at phpMyAdmin and ssh:
Update ub_migrate_map_step_6 Set created_time = ‘2000-01-01 00:00:00’ Where entity_name = ‘customer_entity’; (phpMyAdmin)

/usr/local/bin/php -f bin/ubdatamigration run --step=6 --mode=update (ssh)
PHP Error[2]: Creating default object from empty value
in file /home/joomjiretail/public_html/pub/ub-tool/protected/controllers/Step6Controller.php at line 397
#0 /home/joomjiretail/public_html/pub/ub-tool/protected/controllers/Step6Controller.php(170): Step6Controller->_migrateCustomers()
#1 /home/joomjiretail/public_html/pub/ub-tool/protected/commands/RunCommand.php(74): Step6Controller->actionRun()
#2 /home/joomjiretail/public_html/pub/ub-tool/protected/commands/RunCommand.php(33): RunCommand->_migrateData()
#3 unknown(0): RunCommand->actionIndex()
#4 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/console/CConsoleCommand.php(172): ReflectionMethod->invokeArgs()
#5 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/console/CConsoleCommandRunner.php(71): RunCommand->run()
#6 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/console/CConsoleApplication.php(92): CConsoleCommandRunner->run()
#7 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/base/CApplication.php(185): CConsoleApplication->processRequest()
#8 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/yiic.php(33): CConsoleApplication->run()
#9 /home/joomjiretail/public_html/pub/ub-tool/protected/ubdatamigration_cli.php(5): require_once()
#10 /home/joomjiretail/public_html/bin/ubdatamigration(4): require_once()
 
Please let me know how I can fix this so I can do a full migration of the customers as we need to go live this week.
Credentials are the same from what I sent on this ticket privately a couple weeks ago.
Thanks

#90
Profile photo of Mall Staff 184060.00 $tone June 8, 2020
Public

Hi there,

PHP Error[2]: Creating default object from empty value
in file /home/joomjiretail/public_html/pub/ub-tool/protected/controllers/Step6Controller.php at line 397

That issue because you did deleted at least one migrated customer in M2 backend after the first migration.
Let’s try with steps as following:
+ Run following SQls in your M2 database:

DELETE FROM ub_migrate_map_step_6 WHERE entity_name = 'customer_group' AND m2_id NOT IN (Select customer_group_id From customer_group);
DELETE FROM ub_migrate_map_step_6 WHERE entity_name = 'customer_entity' AND m2_id NOT IN (Select entity_id From customer_entity);
DELETE FROM ub_migrate_map_step_6_customer_address WHERE entity_name = 'customer_address_entity' AND m2_id NOT IN (Select entity_id From customer_address_entity);

+ Once done, clean cache of our module by run command:
rm -rf pub/ub-tool/protected/runtime/cache/
+ And then, you continue with delta migration in step #6 by run CLI command:
/usr/local/bin/php -f bin/ubdatamigration run --step=6 --mode=update
Regards,
Mall.

#92
Profile photo of ftoobi 790.00 $tone June 9, 2020
Public

Hi,
I did what you recommended and that seemed to work but now I am getting the following error when I run step #8:
PHP Error[2]: Creating default object from empty value
in file /home/joomjiretail/public_html/pub/ub-tool/protected/controllers/Step8Controller.php at line 2118
#0 /home/joomjiretail/public_html/pub/ub-tool/protected/controllers/Step8Controller.php(515): Step8Controller->_migrateNewsletterSubscribers()
#1 /home/joomjiretail/public_html/pub/ub-tool/protected/commands/RunCommand.php(74): Step8Controller->actionRun()
#2 /home/joomjiretail/public_html/pub/ub-tool/protected/commands/RunCommand.php(33): RunCommand->_migrateData()
#3 unknown(0): RunCommand->actionIndex()
#4 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/console/CConsoleCommand.php(172): ReflectionMethod->invokeArgs()
#5 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/console/CConsoleCommandRunner.php(71): RunCommand->run()
#6 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/console/CConsoleApplication.php(92): CConsoleCommandRunner->run()
#7 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/base/CApplication.php(185): CConsoleApplication->processRequest()
#8 /home/joomjiretail/public_html/pub/ub-tool/yii-1.1.19/yiic.php(33): CConsoleApplication->run()
#9 /home/joomjiretail/public_html/pub/ub-tool/protected/ubdatamigration_cli.php(5): require_once()
#10 /home/joomjiretail/public_html/bin/ubdatamigration(4): require_once()
Please let me know what queries I need to run so I can finish the delta migration.
Thanks

#93
Profile photo of Mall Staff 184060.00 $tone June 9, 2020
Public

Hi there,

PHP Error[2]: Creating default object from empty value
in file /home/joomjiretail/public_html/pub/ub-tool/protected/controllers/Step8Controller.php at line 2118

Please try with these steps:

+ Run the following SQL queries in your M2 database:

DELETE FROM ub_migrate_map_step_8_review WHERE entity_name = 'review' AND m2_id NOT IN (Select review_id From review);
DELETE FROM ub_migrate_map_step_8_subscriber WHERE entity_name = 'newsletter_subscriber' AND m2_id NOT IN (Select subscriber_id From newsletter_subscriber);
DELETE FROM ub_migrate_map_step_8 WHERE entity_name = 'tax_class' AND m2_id NOT IN (SELECT class_id FROM tax_class);
DELETE FROM ub_migrate_map_step_8 WHERE entity_name = 'tax_calculation_rule' AND m2_id NOT IN (SELECT tax_calculation_rule_id FROM tax_calculation_rule);
DELETE FROM ub_migrate_map_step_8_cms WHERE entity_name = 'cms_block' AND m2_id NOT IN (Select block_id From cms_block);
DELETE FROM ub_migrate_map_step_8_cms WHERE entity_name = 'cms_page' AND m2_id NOT IN (Select page_id From cms_page);

+ Once done, clean the cache of our module by running the command:
rm -rf pub/ub-tool/protected/runtime/cache/
+ And then, you continue with the delta migration in step #8 using this CLI command:
/usr/local/bin/php -f bin/ubdatamigration run --step=8 --mode=update

Regards,
Mall.

#94

Please login or Register to Submit Answer

Written By

Comments