Delta Migration has been running forever.

Hi there,

Please try the steps below:

1. Upgrade to the latest UB Data Migration Pro v3.2.5
2. Then, run the following SQL query in your M2 database:

UPDATE `ub_migrate_map_step_5` SET `created_time` = '2000-01-01 00:00:00' Where entity_name = 'catalog_product_entity';

3. Once done, run the delta migration in step #5 using the CLI command:

php -f bin/ubdatamigration run --step=5 --mode=update

And let me how it goes then.

Regards,
Mall.
 
 

Hi there,

delta migration is running.

 
But not effect in table

That is strange. Please re-correct databases settings in the configuration file of our module at path:
pub/ub-tool/protected/config/db.php
Let’s make sure that, the credentials of the param named ‘db’ are current your M2 database.
Regards,
Mall.

Hi there,

i am sure it is correct.
and Delta step 2, 3, 4 was successful.
Please check step 5 issue carefully.

So lets provide me information about your instance:
+ Admin credentials and URL to the backend of your M2
+ SSH credentials and lets me know the path to M2 folder
+ phpMyadmin URL which can view your M1 and M2 database
I will help to check further and get back to you then.
Regards,
Mall.

Hi there,
I am sorry for replying to you this late as our team has just been back to work from our weekend. 

for 5hrs, only this value was changed. 
 

That occurred because you were running step #5 with the ‘update’ mode and it only updated products which had been changed and newly added products in M1.

I saw a total of products in your M1: 227789 products: http://prntscr.com/vsjnbv
and our migration tool had migrated a total of 222354 products: http://prntscr.com/vsjn47

2020/11/26 01:59:48 -[Processing][delta] Step #5 migration completed with 100 Products;

I checked the migration log of our module and saw that the Step #5 was stopped after that.

And I saw the status ‘Migrating’ in step #5: http://prntscr.com/vsjme8
 
So, please continue with the delta migration in step #5 at your end by running the CLI command:

php -f bin/ubdatamigration run --step=5 --mode=update

and after you finish step #5, please reindex the data again and check the issue you mentioned. And let me how it goes then.

Regards,
Mall.
 

Hi there,

but sometimes, the MySQL connection was broken.
While processing for 3days, If the MySQL connection was broken, and I run the command again.
php -f bin/ubdatamigration run --step=5 --mode=update
Product migration starts with the first product again.
So, We can’t complete this migration step 5.

I have seen your case. Please try steps below to continue with the delta migration in step #5:

1. Open the PHP file at pub/ub-tool/protected/controllers/Step5Controller.php
and find the code lines:

if ($this->runMode == UBMigrate::RUN_MODE_DELTA) {
    $lastMigrationTime = UBMigrate::getLastMigrationTime($this->stepIndex, 'catalog_product_entity');
    $condition .= " AND updated_at >= '{$lastMigrationTime}'";
    //get all product IDs need to delta
    if (!$this->strDeltaProductIds) {
        $this->strDeltaProductIds = $this->_getAllDeltaProductIds($condition);
    }
}

and replace it with: 

//if ($this->runMode == UBMigrate::RUN_MODE_DELTA) {
    $lastMigrationTime = UBMigrate::getLastMigrationTime($this->stepIndex, 'catalog_product_entity');
    $condition .= " AND updated_at >= '{$lastMigrationTime}'";
    //get all product IDs need to delta
    if (!$this->strDeltaProductIds) {
        $this->strDeltaProductIds = $this->_getAllDeltaProductIds($condition);
    }
//}

2. And then, you continue with the delta migration in step #5 by running the CLI command:

php -f bin/ubdatamigration run --step=5

Note: you don’t need to use the param “--mode=update” at this phase. And you need to wait until you finish step #5.

If your server is broken again, you can continue step #5 with that CLI command (without “--mode=update”). 

Regards,
Mall.

Hi there,

is there a table that keeps track of what product ID it is on that we could maybe manipulate and insert a record to say the last product ID?

You can check the data records in the table ‘catalog_product_entity’ in your M2 database:

catalog_product_entity.updated_at (if some of the migrated products had changed information in your M1 after the delta migration, the value of this field in those products will be in your M2 too)

catalog_product_entity.entity_id (after you finished the delta migration in step #5, you will see the new ids of the newly added products in M1 after the first migration)

We need delta migration (not from first product)

Please note that our migration tool scans from the first product to run delta update for both the migrated products in the first migration.

Currently, Step5 is running.  but will take 1~2days to complete.

If you did the steps suggested in my reply #10, it means the migration process is continuing in step #5. Actually, the migration time depends on the volume of your products in M1 and your server’s performance.

Do you want to run delta update for both the migrated products in the first migration? Or you just want to run delta migration for the newly added products only in M1 after the first migration?
 
Regards,
Mall.
 
 

Hi there,

I have checked your instance and seen the issue you mentioned. Then I closed the tweaked code suggested in my reply #10: http://prntscr.com/vtzyzl (it is not needed at this phase).

Now, to solve the issue, you can try the steps below:

1. Run the following SQL query in your M2 database:

UPDATE `ub_migrate_map_step_5` SET `offset` = 222300 Where entity_name = 'catalog_product_entity';

2. Once done, run delta migration in step #5 using the CLI command:

php -f bin/ubdatamigration run --step=5

Note: you don’t need to use the param “--mode=update” at this phase. And make sure that you wait until you finish step #5.
 
In case your server is broken again, you can simply continue step #5 with that CLI command (without “--mode=update”). 
 
Regards,
Mall.