Error during Step 7 Order Import

Hi there,

[error] [ub_data_migration] Mage2SalesQuoteItemOption: Code cannot be blank. 

That issue because the value of the code field is required in Magento2’s database but some records in the table sales_flat_quote_item_option was missed valued on the code field in your Magento1’s database.
To solve that issue, you have to clean some records which are missed value on the code field in the table ‘sales_flat_quote_item_option’. To delete that records, you could use SQL statement as followings:

Delete From sales_flat_quote_item_option Where code = '' OR code IS NULL;

Once done, you could continue with data migration in the step #7 of our tool.

M2 version: 2.2

What version of our tool in your working now? For compatible with Magento ver.2.2.0, you must to install ver.3.1.1 of our tool.
Regards,
Mall.
 
 

Hi there, 
Please give us sometime to follow up your ticket. 
It’s our development team’s night time, while you have our office IPs whitelisted. We will check and get back to you tomorrow right after we’re back to our office. 
Thanks for your understanding and patience.
Regards,
Ubertheme team

Hi there,

[error] [ub_data_migration] Mage2SalesQuoteItemOption: Code cannot be blank. 

That issue related to the step #7 of our tool. However when I checked your Magento2 instance, I see you have reset data migration in the step #5 of our tool:
https://prnt.sc/h23je5
So, you should continue from the step 5 as normal. If you still run across any issues then, kindly pause the process and provide me details of the issue. I will check further and help you out.
Regards,
Mall.

Hi there,

You may log in and check it again now.

Yeah, let’s pause all your activities at your end now. I will check further and get back you soon.
Regards,
Mall.

Hi there,

 [error] [ub_data_migration] Mage2SalesQuoteItemOption: Code cannot be blank. 

To handle for that case, I have added more tweak code in the step #7 of our tool at:
pub/ub-tool/protected/controllers/Step7Controller.php/
However after I continued with data migration in the step #7, I run across new issue with your data:

Message: CDbCommand failed to execute the SQL statement: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry ‘625096’ for key ‘PRIMARY’. The SQL statement executed was: INSERT INTO `quote_item_option` (`option_id`, `item_id`, `product_id`, `code`, `value`) VALUES (:yp0, :yp1, :yp2, :yp3, :yp4)

It seems you have some sample data in the sales data section before using our tool to migrate? Please confirm if this is true under your case? 
And I also see you selected keep original IDs in the step #7 of our tool: https://prnt.sc/h28acd
Please note this option requires your Magento 2 must be a fresh instance only.
For now, please reset the step #7 and re-start data migration in this step. And let me know how it goes.
Regards,
Mall.

Hi there,

Also, I tried logging into a customer account on the frontend, but it is not letting me login. I am 100% sure I was using the correct password.

Could you please check once again the credentials, and make sure it can be used to log in successfully in M1 db? If the issue still persists, then let me know. Other completed migration projects using our tool did not run across such login issue.

But when I try to view an order, I am getting an error:
Exception #0 (InvalidArgumentException): Unable to unserialize value.
This doesn’t happen with every order, but many of them. Two examples are orders #300002605 and #100150447

I had a quick check and see the issue was due to the changes of Magento ver.2.2.0 (serialize/json enconde function) in some sales data section.
We’re checking this again and will add the tweak code in the step 7. Once done, I will get back to you with the workaround.
Thanks for your patience.
Regards,
Mall.

Hi there,

 But when I try to view an order, I am getting an error:
Exception #0 (InvalidArgumentException): Unable to unserialize value.
This doesn’t happen with every order, but many of them. Two examples are orders #300002605 and #100150447

To handle for that cases, we have added more tweak code in the step #7 of our tool at:
pub/ub-tool/protected/controllers/Step7Controller.php
To apply this workaround, kindly follow steps below:
+ Step 1: Download the php file HERE
+ Step 2: Upload the file you have downloaded to the path pub/ub-tool/protected/controllers/Step7Controller.php
+ Step 3: Kindly reset data migration in the step #8, #7 of our tool and restart data migration in these steps.
Regarding the login issue, please provide me credentials of such customer so that I could debug further.
Regards,
Mall.
 
 

Hi there,
Yeah, let’s give me more time to check further on customer login issue in your working. And i will get back you soon.
Regards,
Mall.

Hi there,

Regarding the login issue, please provide me credentials of such customer so that I could debug further.
[email protected]
314000

Kindly re-check, I got similar issue as you mentioned in the Google Chrome browser. I have tried login with that customer credentials in Firefox browser and I see it was working fine: http://i.prntscr.com/fVV_lUieSRyb-PKjQmeZPQ.png
Please try to clean cache and old cookies data in your Google Chrome browser. And re-login with such credentials again, and let me know how it goes.
PS. I did the same on Chrome browser and can log in as normal now: https://prnt.sc/h2m1il
Regards,
Mall.

Hi there,

Did you make any changes or was the problem just the browser cookies/cache?

For your information, in your Magento instance, I have changed the folder/files permission for var/cache and var/tmp folders:
chmod -R 777 var/tmp/
chmod -R 777 var/cache
Regards,
Mall.

Hi there,
As you have big volume of data, it might take longer. Please continue the process though.
Also please consider upgrading your server to improve the performance for next migration task.
Regards,
Mall.
 

Hi there,

I have finished resetting and re-running step #7 and #8. Unfortunately, the serialize error still exists when I try to view orders.

Did you do update the new code file in the step #7 as per my suggesting in the reply #13?
Regards,
Mall.

Hi there,
Yeah, i have rechecked further in your database and i see more context on that issue: https://prnt.sc/h303xp
I will provide you more tweak code in the step #7 of our tool to solve that soon.
Thanks for your waiting.
Regards,
Mall.

Hi there,

ok, I am resetting steps #7 and #8 again now

Ok, after you have reset successfully in the step #7, kindly do steps as followings once more:
+ Step 1: Download the php file HERE (i have added more new tweak code in this file)
+ Step 2: Upload the file you have downloaded to the path pub/ub-tool/protected/controllers/Step7Controller.php
+ Step 3: Start data migration in the step #7
And tell me know how it goes.

Regards,
Mall.

Hi there,
For your information, if your M2 instance is a fresh instance before use our tool. You could select ‘KEEP ORIGINAL IDS’ option in the step #7 to improve migration time in this step.
Regards,
Mall.
 

Hi there,

Unfortunately, the serialize error still exists

Kindly have a look again, i have rechecked in your instance and see that is another issue’s type:

1 exception(s):
Exception #0 (Exception): Warning: json_decode() expects parameter 1 to be string, array given in /chroot/home/mtwotact/m2.tacticalshit.com/html/vendor/magento/framework/Serialize/Serializer/Json.php on line 37

Exception #0 (Exception): Warning: json_decode() expects parameter 1 to be string, array given in /chroot/home/mtwotact/m2.tacticalshit.com/html/vendor/magento/framework/Serialize/Serializer/Json.php on line 37
#0 [internal function]: Magento\Framework\App\ErrorHandler->handler(2, 'json_decode() e...', '/chroot/home/mt...', 37, Array)
#1 /chroot/home/mtwotact/m2.tacticalshit.com/html/vendor/magento/framework/Serialize/Serializer/Json.php(37): json_decode(Array, true)
#2 /chroot/home/mtwotact/m2.tacticalshit.com/html/vendor/magento/module-bundle/Block/Adminhtml/Sales/Order/View/Items/Renderer.php(158): Magento\Framework\Serialize\Serializer\Json->unserialize(Array)

That because Magento2 has double call json_encode in the value of the ‘bundle_selection_attributes’ in product options in sale order items of bundle products.
To handle that, we have added more tweak code in the step #7 of our tool. Kindly do steps as followings:
+ Step 1: Download the php file HERE (i have added more new tweak code in this file)
+ Step 2: Upload the file you have downloaded to the path pub/ub-tool/protected/controllers/Step7Controller.php
+ Step 3: Run delta migration in the step #7 by using CLI command: (you mustn’t reset data migration in step #7 at this time)
php -f bin/ubdatamigration run --step=7 --mode=update
And tell me know how it goes.
Thank you for your understanding.
Regards,
Mall.

Hi there,

I have replaced the Step7Controller.php file and ran the delta migration for step #7. Unfortunately the “Unable to unserialize value” error still exists when trying to view an order

Kindly re-check, i have just try open a sales order which has issue as you mentioned in first time and now i see it working fine: http://i.prntscr.com/L7kpbeJdRzObRxJjeta_RA.png

 as well as when trying to add a product to a category.

Can you tell me more details information about that? Did you couldn’t add new product?

Also, I notice the Product Price index gets stuck when trying to reindex.
https://prnt.sc/h43i0g

Let’s try reindex your data by run CLI commands as followings:
php -f bin/magento indexer:reset
php -dmemory_limit=-1 -f bin/magento indexer:reindex
php -f bin/magento cache:clean
And tell me know how it goes.
Regards,
Mall.

Hi there,
Kindly pause all your activities at your end. I will check further in your working and reply you more later.
Regards,
Mall.

Hi there,
I have tried reindex data in your instance and i see it stuck on processing in long time with Product Price indexer with no error message too.
I have checked further in your server via SSH and i see this:
[mtwotact@sip4-859 public_html]$ php -i | grep ‘memory_limit’
memory_limit => 768M => 768M
Kindly contact to your system admin or hosting provider to increase the value of php’s setting: memory_limit = 2048M or higher 
For your information: I have re-setting the migrated root category for current default store of default website in your Magento2 site too.
Let’s contact to changes php’s settings of your server and restart web server after your changing.
And then, retry with index data commands:
php -f bin/magento indexer:reset
php -dmemory_limit=-1 -f bin/magento indexer:reindex
php -f bin/magento cache:clean
And tell me know how it goes.
Regards,
Mall.
 

Hi there,

 Unfortunately the indexer still gets stuck on Product Price

That is strange, as you see Magento does not raises any error message on that.
I’m stay at home now and i can’t check more in your instance now because your whitelist settings.
Regards,
Mall.
 
 

Hi there,
I have checked further in your instance now. Actually your server processed some CLI commands too slowly. 
I tried to reindex the catalog_product_price indexer only, but it keeps processing without any error message from Magento’s system.
I also have run some chmod commands but it tooks too long time to finish.
find . -type d -exec chmod 775 {} +;
find . -type f -exec chmod 664 {} +;
Can you let me know how many CPU’s cores in your server now? Can you upgrade Ram and CPU for better performance?
Meanwhile, I also checked and see your memory now is 768M (see: http://i.prntscr.com/tN_m3YOyRZuWiSqZ17U_rQ.png) not 2048M as you mentioned. 

I had the memory increased to 2048M and tried those commands.

Regards,
Mall.
 
 

Hi there,

You can check http://m2.tacticalshit.com/info.php for all info.
You can see that it currently says memory_limit 2048M

That is the setting for Apache web server only. You need to request to change setting for PHP in CLI as well.
Also, please answer my question in my previous reply:

How many CPU’s cores in your server now? Can you upgrade RAM and CPU for better performance? 

Regards,
Mall.

Hi there,

https://prnt.sc/h4mxpr

Yes, I see you have 2 cores CPU 1.9Ghz. Kindly contact to your hosting provider to request increase the PHP’s memory_limit on CLI environment as well.
And please ask your hosting provider for further help about why we could not process when running reindex for indexer ID: catalog_product_price
php -f bin/magento indexer:reset catalog_product_price
php -f bin/magento indexer:reindex catalog_product_price
As you see, I already enabled full debug mode but Magento did not raise any errors associated with that.
At present, I doubt about the issue of PHP CLI’s memory limit. Normally, in a similar case like this, PHP will notify the memory exceeded issue. Yet, no error message under your circumstance. 
Beside, your server CPU power might be another factor that affects this. 
Regards,
Mall.

Hi there,
At this phase, based on the fact I explained so far, the issue shows indexer-related issue. I see no clue about ‘Unable to Unserialize value’ at present.
Please consider asking your hosting provider for further help on the server side.
Further, please make sure you also request to change the setting for PHP in CLI as mentioned in my previous reply:

That is the setting for Apache web server only. You need to request to change setting for PHP in CLI as well.

Regards,
Mall.

Hi there,

After the successful reindex, I tried opening some orders again and I am still not able to open them and the ‘Unable to Unserialize value’ error shows in the exception log.

I have checked further on your instance and see the details of that issue when view the order with ID = 100150446 as below:

[2017-11-02 02:58:25] main.CRITICAL: Unable to unserialize value. {“exception”:”[object] (InvalidArgumentException(code: 0): Unable to unserialize value. at /chroot/home/mtwotact/m2.tacticalshit.com/html/vendor/magento/framework/Serialize/Serializer/Json.php:39)”} []
[2017-11-02 03:05:44] main.CRITICAL: Unable to unserialize value: a:1:{s:15:”authorize_cards”;a:1:{s:32:”91ebbdd5385d934250d3d4c084b0cb1e”;a:14:{s:2:”id”;s:32:”91ebbdd5385d934250d3d4c084b0cb1e”;s:16:”requested_amount”;s:0:””;s:15:”balance_on_card”;s:0:””;s:13:”last_trans_id”;s:11:”60734202780″;s:16:”processed_amount”;s:6:”208.90″;s:7:”cc_type”;s:2:”VI”;s:8:”cc_owner”;N;s:8:”cc_last4″;s:4:”7843″;s:12:”cc_exp_month”;s:1:”5″;s:11:”cc_exp_year”;s:4:”2020″;s:11:”cc_ss_issue”;N;s:17:”cc_ss_start_month”;N;s:16:”cc_ss_start_year”;N;s:15:”captured_amount”;s:6:”208.90″;}}}

That because your sales data still has some serialized data has not convert to json data yet. I will help you to detect further that issue in your instance. And i will get back you later.
Regards,
Mall.
 

Hi there,

(InvalidArgumentException(code: 0): Unable to unserialize value. at /chroot/home/mtwotact/m2.tacticalshit.com/html/vendor/magento/framework/Serialize/Serializer/Json.php:39)”} []
[2017-11-02 03:05:44] main.CRITICAL: Unable to unserialize value: a:1:{s:15:”authorize_cards”;a:1:{s:32:”91ebbdd5385d934250d3d4c084b0cb1e”;a:14:{s:2:”id”;s:32:”91ebbdd5385d934250d3d4c084b0cb1e”;s:16:”requested_amount”;s:0:””;s:15:”balance_on_card”;s:0:””;s:13:”last_trans_id”;s:11:”60734202780″;s:16:”processed_amount”;s:6:”208.90″;s:7:”cc_type”;s:2:”VI”;s:8:”cc_owner”;N;s:8:”cc_last4″;s:4:”7843″;s:12:”cc_exp_month”;s:1:”5″;s:11:”cc_exp_year”;s:4:”2020″;s:11:”cc_ss_issue”;N;s:17:”cc_ss_start_month”;N;s:16:”cc_ss_start_year”;N;s:15:”captured_amount”;s:6:”208.90″;}}}

I have detected further in your instance and see that issue come from value of the ‘additional_information’ field in the table named
‘sales_order_payment’
This is order record has that issue: https://prnt.sc/h540a1
And here is associated recordin the table sales_order_payment which was raised error:
https://prnt.sc/h540mc
Magento ver2.2.0 was changed method from serialized() to json_encode() to encode value of that field too. Therefor we have to add more tweak code to convert that values in step #7 of our tool.
So, to handle for that case, you could do steps as followings:
+ Step 1: Download the php file HERE (i have added more new tweak code in this file)
+ Step 2: Upload the file you have downloaded to the path pub/ub-tool/protected/controllers/Step7Controller.php
+ Step 3: Run delta migration in the step #7 by using CLI command: (you mustn’t reset data migration in step #7 at this time)
php -f bin/ubdatamigration run --step=7 --mode=update
And tell me know how it goes.
Thank you for your understanding.
Regards,
Mall.
 

Hi there,

Exception #0 (InvalidArgumentException): Unable to unserialize value: #0 /chroot/home/mtwotact/m2.tacticalshit.com/html/vendor/magento/framework/Model/ResourceModel/AbstractResource.php(147): Magento\Framework\Serialize\Serializer\Json->unserialize(‘a:1:{s:19:”real…’) #1 /chroot/home/mtwotact/m2.tacticalshit.com/html/vendor/magento/framework/Model/ResourceModel/Db/AbstractDb.php(505): Magento\Framework\Model\ResourceModel\AbstractResource->_unserializeField(Object(Magento\Sales\Model\Order\Payment\Transaction), ‘additional_info…’, Array)

As you see that issue come from another context which related value of additional_info filed in the sales payment transaction.
Let’s me check further and tell you how to handle more for that case later.
Regards,
Mall.

Hi there

Exception #0 (InvalidArgumentException): Unable to unserialize value: #0 /chroot/home/mtwotact/m2.tacticalshit.com/html/vendor/magento/framework/Model/ResourceModel/AbstractResource.php(147): Magento\Framework\Serialize\Serializer\Json->unserialize(‘a:1:{s:19:”real…’) 

This is root of that issue: https://prnt.sc/h5789g
To handle that, you could do steps as followings:
+ Step 1: Download the php file HERE (i have added more new tweak code in this file)
+ Step 2: Upload the file you have downloaded to the path pub/ub-tool/protected/controllers/Step7Controller.php
+ Step 3: Run delta migration in the step #7 by using CLI command: (you mustn’t reset data migration in step #7 at this time)
php -f bin/ubdatamigration run --step=7 --mode=update
And tell me know how it goes.
Regards,
Mall.

Hi there,

I placed a test order 100150448 on M1 site and left it in Processing status. I then ran Delta Step7 on M2 from admin panel. The order imported correctly. I then shipped the order on M1 and it was then in Complete status. I ran Delta Step7 again from admin panel, but the order in M2 did not update to Complete and is still in Processing status

To update information for migrated sales orders, you must to run delta migration with --mode=update:
php -f bin/ubdatamigration run --step=7 --mode=update
Regards,
Mall.

Hi there,

I just wish I could figure out the slow reindexing issue. It only takes 10 minutes to reindex our store in M1, but in M2 the Product Price index is taking 3+ hours or even 24+ hours. I have no idea if the problem is related to the migration that your extension performed or something else, but if you have any idea what the problem is with the indexer, let me know.

With your information provided above (reply #48), you can see, that issue related to Magento 2’s issue. And our tool doesn’t affect reindex product price data function. 
Thanks for your understanding.
Regards,
Mall.