Hello.
We are using
UB Data Migration Pro (CE) -- 3.2.4
We run “php -f bin/ubdatamigration run --step=5 --mode=update” as Delta migration.
But it took 7days.. forever running.. not finish.
The main issue is First that this migration doesn’t work with delta. it is working from initial data.
The main issue is Second that when the MySQL gone issue, If we run “php -f bin/ubdatamigration run --step=5 --mode=update” again, it start from 1st Product. not Continue.
This is ub_migrate_step table.
This is ub_migrate_map_step_5 table
All item created_time is 2020-10-12 (Old time)
Please check this critical issue.
Thank you.
14 answers
Hi there,
Please try the steps below:
1. Upgrade to the latest UB Data Migration Pro v3.2.5
2. Then, run the following SQL query in your M2 database:
UPDATE `ub_migrate_map_step_5` SET `created_time` = '2000-01-01 00:00:00' Where entity_name = 'catalog_product_entity';
3. Once done, run the delta migration in step #5 using the CLI command:
php -f bin/ubdatamigration run --step=5 --mode=update
And let me how it goes then.
Regards,
Mall.
Hello.
We did as you mentioned.
But Same Issue.
delta migration is running.
But not effect in table
of db.
Please check this issue.
Thank you.
Hi there,
delta migration is running.
But not effect in table
That is strange. Please re-correct databases settings in the configuration file of our module at path:
pub/ub-tool/protected/config/db.php
Let’s make sure that, the credentials of the param named ‘db’ are current your M2 database.
Regards,
Mall.
Yes. we already did. I checked it.
i am sure it is correct.
and Delta step 2, 3, 4 was successful.
Please check step 5 issue carefully.
thank you.
Hi there,
i am sure it is correct.
and Delta step 2, 3, 4 was successful.
Please check step 5 issue carefully.
So lets provide me information about your instance:
+ Admin credentials and URL to the backend of your M2
+ SSH credentials and lets me know the path to M2 folder
+ phpMyadmin URL which can view your M1 and M2 database
I will help to check further and get back to you then.
Regards,
Mall.
Hello.
yet, migration is running with step5.
for 5hrs, only this value was changed.
from 0 to 58600
Hope this information would be helpful for you.
thank you
Hi there,
I am sorry for replying to you this late as our team has just been back to work from our weekend.
for 5hrs, only this value was changed.
That occurred because you were running step #5 with the ‘update’ mode and it only updated products which had been changed and newly added products in M1.
I saw a total of products in your M1: 227789 products: http://prntscr.com/vsjnbv
and our migration tool had migrated a total of 222354 products: http://prntscr.com/vsjn47
2020/11/26 01:59:48 -[Processing][delta] Step #5 migration completed with 100 Products;
I checked the migration log of our module and saw that the Step #5 was stopped after that.
And I saw the status ‘Migrating’ in step #5: http://prntscr.com/vsjme8
So, please continue with the delta migration in step #5 at your end by running the CLI command:
php -f bin/ubdatamigration run --step=5 --mode=update
and after you finish step #5, please reindex the data again and check the issue you mentioned. And let me how it goes then.
Regards,
Mall.
Hello!
I am sorry that you couldn’t understand our issue.
We need delta migration with update mode.
php -f bin/ubdatamigration run --step=5 --mode=update
But it runs processing from the first product (all product- not working with delta migration mode)
so, it will take 3days to complete all 227789 products.
but sometimes, the MySQL connection was broken.
While processing for 3days, If the MySQL connection was broken, and I run the command again.
php -f bin/ubdatamigration run --step=5 --mode=update
Product migration starts with the first product again.
So, We can’t complete this migration step 5.
If you would have any question, please let me know
Thank you.
Hi there,
but sometimes, the MySQL connection was broken.
While processing for 3days, If the MySQL connection was broken, and I run the command again.
php -f bin/ubdatamigration run --step=5 --mode=update
Product migration starts with the first product again.
So, We can’t complete this migration step 5.
I have seen your case. Please try steps below to continue with the delta migration in step #5:
1. Open the PHP file at pub/ub-tool/protected/controllers/Step5Controller.php
and find the code lines:
if ($this->runMode == UBMigrate::RUN_MODE_DELTA) {
$lastMigrationTime = UBMigrate::getLastMigrationTime($this->stepIndex, 'catalog_product_entity');
$condition .= " AND updated_at >= '{$lastMigrationTime}'";
//get all product IDs need to delta
if (!$this->strDeltaProductIds) {
$this->strDeltaProductIds = $this->_getAllDeltaProductIds($condition);
}
}
and replace it with:
//if ($this->runMode == UBMigrate::RUN_MODE_DELTA) {
$lastMigrationTime = UBMigrate::getLastMigrationTime($this->stepIndex, 'catalog_product_entity');
$condition .= " AND updated_at >= '{$lastMigrationTime}'";
//get all product IDs need to delta
if (!$this->strDeltaProductIds) {
$this->strDeltaProductIds = $this->_getAllDeltaProductIds($condition);
}
//}
2. And then, you continue with the delta migration in step #5 by running the CLI command:
php -f bin/ubdatamigration run --step=5
Note: you don’t need to use the param “--mode=update” at this phase. And you need to wait until you finish step #5.
If your server is broken again, you can continue step #5 with that CLI command (without “--mode=update”).
Regards,
Mall.
Hello.
Yes. i did.
But yet, only this field is changed.
is there a table that keeps track of what product ID it is on that we could maybe manipulate and insert a record to say the last product ID?
We need delta migration (not from first product)
Currently, Step5 is running. but will take 1~2days to complete.
Thank you.
Hi there,
is there a table that keeps track of what product ID it is on that we could maybe manipulate and insert a record to say the last product ID?
You can check the data records in the table ‘catalog_product_entity’ in your M2 database:
catalog_product_entity.updated_at (if some of the migrated products had changed information in your M1 after the delta migration, the value of this field in those products will be in your M2 too)
catalog_product_entity.entity_id (after you finished the delta migration in step #5, you will see the new ids of the newly added products in M1 after the first migration)
We need delta migration (not from first product)
Please note that our migration tool scans from the first product to run delta update for both the migrated products in the first migration.
Currently, Step5 is running. but will take 1~2days to complete.
If you did the steps suggested in my reply #10, it means the migration process is continuing in step #5. Actually, the migration time depends on the volume of your products in M1 and your server’s performance.
Do you want to run delta update for both the migrated products in the first migration? Or you just want to run delta migration for the newly added products only in M1 after the first migration?
Regards,
Mall.
Hi there.
Now, The delta migration was completed.
But new products of M1 were not migrated to M2.
There are old products from the first migration.
M1 has 227790 products total
But M2 only shows 222355.
We already did reindexer.
Please check this issue.
Thank you.
Hi there,
I have checked your instance and seen the issue you mentioned. Then I closed the tweaked code suggested in my reply #10: http://prntscr.com/vtzyzl (it is not needed at this phase).
Now, to solve the issue, you can try the steps below:
1. Run the following SQL query in your M2 database:
UPDATE `ub_migrate_map_step_5` SET `offset` = 222300 Where entity_name = 'catalog_product_entity';
2. Once done, run delta migration in step #5 using the CLI command:
php -f bin/ubdatamigration run --step=5
Note: you don’t need to use the param “--mode=update” at this phase. And make sure that you wait until you finish step #5.
In case your server is broken again, you can simply continue step #5 with that CLI command (without “--mode=update”).
Regards,
Mall.