Big Catalog

Hello
 
My catalog is arround 11000+ products
since 24hours only 4500 products are imported. Is this normal or have i some trouble ?
 
Regards

37 answers

Profile photo of ubdev Staff 98590.00 $tone September 12, 2018
Public

Hi there, 
Could you please let us know your server capacity? As mentioned in our Magento 2 data migration FAQs here, it might take about 1.5 or 2 hours to migrate 16,000 products (supposed that you are on a VPS / Cloud server with a high performance SSD hard drives, Ram from 16 GB+, quad-core CPU). 
Meanwhile, please also check our migration tool’s log at: pub/ub-tool/protected/runtime/ub_data_migration.log and let us know if there’s any error notices in place.
Regards,
Ubertheme team
 

#1
Profile photo of maximesabourin 270.00 $tone September 12, 2018
Public

Hi
My server is a VPS (Proxmox CT)
6 cores and 48G of RAM
Normal (non ssd) disk
 
The file pub/ub-tool/protected/runtime/ub_data_migration.log doesn’t exist
Migration is launched by CLI
 
Regards
Maxime Sabourin

#2
Profile photo of ubdev Staff 98590.00 $tone September 12, 2018
Public

Hi Maxime Sabourin, 
Please provide us the following credentials, we will check further and get back to you then: 

  • Admin url of your M2 instance
  • Admin and SSH credentials of your M2 instance
  • Let us know the path of your web root folder

PS. Please mark your reply ‘private’, then it’s safe to share your site credentials in this thread, only you and our technical team can access.
Regards,
Ubertheme team

#3
Profile photo of Mall Staff 184060.00 $tone September 13, 2018
Public

Hi there,

the process is still runing 40h ago
9500 products are imported

Are you run data migration via dashboard back-end or CLI commands of our migration tool?
Regards,
Mall.

#10
Profile photo of Mall Staff 184060.00 $tone September 14, 2018
Public

Hi there,
I could access your M2 web root folder via SSH terminal as your instruction.
But when checking again in M2 back-end, I saw that you have finished data migration in all steps of our migration tool:
http://i.prntscr.com/l6n3AQaeTsWE6HIO-jgYIg.png

So, please continue with other required steps as indicated in the Readme.html guide that comes packed with your download package. 
If you run across any issue, please get back to us here. 
Regards,
Mall.

#14
Profile photo of Mall Staff 184060.00 $tone September 17, 2018
Public

Hi there,

BUT
Majority of the link betwwen configurable product and simple product are LOST
example with ring name abigael. a configurable product with 12 associated simple product. in the M2 the product arent linked

I have checked both your M1 and M2 instances and saw the issue on the configurable products you mentioned. This is a strange case, it hasn’t happened with our migration tool before.

I need to check further in your M1 and M2 databases. However, it seems you have disabled the SSH credentials you provided: http://i.prntscr.com/ak9XJCtSQdmaL4O9BO47nQ.png. Please help to reactivate SSH access again. 

Also, if possible, could you provide me the phpMyadmin credentials of your M1 and M2 databases?
Further, did you reindex your data successfully after the migration was done?
Regards,
Mall.

#16
Profile photo of Mall Staff 184060.00 $tone September 17, 2018
Public

Hi there,

I checked further in your M1 and M2 database for the configurable product you mentioned (product has entity Id = 13655), and saw as in the following screenshots:

+ In M1 database: 

+ In M2 migrated database:

  • The configurable product attributes of that product: https://prnt.sc/kva8td
  • And the related attributes: http://i.prntscr.com/lYsvapjOQ32aVXAkhfDL3w.png
  • And the data relationship between the parent and child products: http://i.prntscr.com/Id-B3xVwQO_dZajFE9xlEQ.png

So, you could see all the related product’s data was migrated correctly by our migration tool.
And then, after checking further in the system log, I saw some exceptions which related to product media files as in the screenshots below:
http://i.prntscr.com/aqBMqqldQVysIs2b1qppgQ.png

And that may affect the M2 core functions in the product editing page in your back-end.
So, please move all your product’s image files from M1 instance to M2 instance and re-check the issue you mentioned. 
If the issue still persists, then let me know. 

Regards,
Mall.
 
 
 
 

#19
Profile photo of Mall Staff 184060.00 $tone September 17, 2018
Public

Hi there,

can you check ?

As explained in reply #19. All related data of that product was migrated correct in database. Let’s me more time, I will help to check further in your M2 instance on the issue in back-end view of that configurable product.
Also, you should check more at your end too.
Regards,
Mall.

#21
Profile photo of Mall Staff 184060.00 $tone September 18, 2018
Public

Hi Maxime,

BUT
Majority of the link betwwen configurable product and simple product are LOST
example with ring name abigael. a configurable product with 12 associated simple product. in the M2 the product arent linked

I helped to check further in your instance once again, but I couldn’t find the root of that issue. As explained in my first reply #19, all related data on those products was migrated correctly into your database.

I also checked in some other configurable products, and I saw they’re working fine, without the same issue:

You can re-check those configurable products at your end too.
So, you could see the issue doesn’t related to our migration tool.

Could you please restart data migration all steps in a new M2.2.5 instance one more? If the issue still persists on this new instance, please help to provide me a latest backup of your M1 database, I will run a testing migration at my end to help you out. 
Regards,
Mall.
 

#22
Profile photo of maximesabourin 270.00 $tone September 18, 2018
Public

Hi Mall
I can do this test but

  • it took 3 days to perform last time
  • why would it be different ?

I’m not able to understand the magento database but all that as to be done as be done

  • fresh install
  • media migration

And the result is strange.
somme are good (but not the majority)
 
Regards
 

#23
Profile photo of ubdev Staff 98590.00 $tone September 19, 2018
Public

Hi Maxime,
We know it takes you extra time, yet it’s the best option at present. 
We’re not completely sure if you strictly follow our migration workflow mentioned in our Readme.html that comes packed with your download or not. We noticed that you had import & export csv which might affect the data migration. 
So we highly recommend you follow steps as mentioned in the latest reply #22 of our team member Mall. 
Thanks for your understanding.
Regards,
Ubertheme team

#24
Profile photo of Mall Staff 184060.00 $tone September 21, 2018
Public

Hi Maxime Sabourin,

I wonder if the migration process completes at your end now?

same access as previously if you want to check

I tried to access to check, but it seems you have disabled the admin credentials you provided before. I got the message below:

You did not sign in correctly or your account is temporarily disabled.

Regards,
Mall.
 

#26
Profile photo of Mall Staff 184060.00 $tone September 24, 2018
Public

Hi Maxime Sabourin,

I wonder if the migration process completes at your end now? And how about the issues you mentioned on migrated configurable products?
Regards,
Mall.

#29
Profile photo of ubdev Staff 98590.00 $tone September 24, 2018
Public

Hi Maxime Sabourin, 
As explained in Mall’s replies above, all required product data has been migrated into your database.
The issue when viewing configurable products just occurred on a few specific products. This indicates that this issue was not caused by our migration tool, there should be something from specific data under your circumstance.
Please help to provide us a backup of your latest M1’s db, we will debug further at our end, and let you know once we found the root of issue.
Regards,
Ubertheme team

#31
Profile photo of ubdev Staff 98590.00 $tone September 25, 2018
Public

Hi Maxime Sabourin, 
We’re still proceeding a test migration at our end to debug the root of issue. 
For your information on the current status -- http://i.prntscr.com/5kaSAmK-TC63QqCWhHoQqg.png
We’ll keep you informed later on once we have new finding.
Regards,
Ubertheme team

#34
Profile photo of Mall Staff 184060.00 $tone September 26, 2018
Public

Hi there,

is it faster than at my side ?

I’m not sure, because I didn’t know the migration time at your end.
For your information: I did data migration on your M1 database in my laptop which has 16 GB RAM, Intel® Core™ i7-3520M CPU @ 2.90GHz × 4  and SSD hard disk
And I have finished data migration in the step #5: http://i.prntscr.com/kZwtqmkoQLKOG06hzltyWQ.png
I will continue check further on the issue you mentioned on migrated configurable products and get back you soon.
Regards,
Mall.

#36
Profile photo of Mall Staff 184060.00 $tone September 27, 2018
Public

Hi there,

 BUT
Majority of the link betwwen configurable product and simple product are LOST
example with ring name abigael. a configurable product with 12 associated simple product. in the M2 the product arent linked

I have finished testing data migration with your M1’s database in my local:
http://i.prntscr.com/siB9etX5QiCzxiZsHZyKZg.png
And all migrated products were listed in the back-end: http://i.prntscr.com/v1hnso8pQWSmc2QjQmraeA.png
I have checked the issue on the Edit page of some configurable products and saw that the issue happened on some configurable products as you mentioned. 
I noticed that they refer to the configurable products of the attribute sets: ‘Alliance Migrated’ and ‘Bague Migrated’. Other migrated configurable products are fine.
Then, I detected further in your database and found the root of that issue. It relates to your product data settings in M1 database.
In M2 core function of the Edit page on configurable products, M2 has added filter to get children products which has ‘not required’ options only (required_options = 0). You could find the code line below:

->addFilterByRequiredOptions()

in the PHP core file at: vendor/magento/module-configurable-product/Model/Product/Type/Configurable.php
and the details of the addFilterByRequiredOptions() function:

$this->addAttributeToFilter('required_options', [['neq' => 1]], 'left');

But a few simple children products of some configurable products have the setting ‘required_options’ = 1. This is an incompatibility setting against the new working rules of M2 which caused the issue you mentioned.

You can list those children simple products again in your M1 database at your end by running the following SQL command:

SELECT e.* FROM `catalog_product_entity` as e WHERE e.`required_options` = 1 AND e.`entity_id` IN (SELECT product_id
FROM `catalog_product_super_link` as linktable );

Therefore, to solve this issue, you must re-update the value of the ‘required_options’ filed from ‘1’ to ‘0’ for all children simple products which associated to the configurable products that has such issue. To mass update, you can run the following SQL in your M2 database:

UPDATE `catalog_product_entity` SET `required_options` = 0 WHERE `entity_id` IN (SELECT product_id FROM  `catalog_product_super_link`);

+ Once done, please reindex the migrated data again and clean M2 cache. And the issue should be solved.

For your information, I did that at my end: http://i.prntscr.com/VjIUvIwuRTGWluUyaWXoYQ.png
and here’s an example of the configurable product you mentioned -- it’s working fine in the back-end: http://i.prntscr.com/zI9GCgxJS12Te42CMa1QAw.png
And it also works fine in the front-end view too: http://i.prntscr.com/w_7J1YqsRe6FL6drK5DaAQ.png
And as you can see, the issue relates to the bad settings of your products in M1 database, and does not relate to our migration tool.
Regards,
Mall.

#37

Please login or Register to Submit Answer

Written By

Comments