-
Notifications
You must be signed in to change notification settings - Fork 9.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OOM/Timeouts when saving a category with large catalog as all product URLs are regenerated #7874
Comments
One of my 2.1.4 merchants is experiencing this exact issue. They have a category that contains 538 children/descendent categories and those child categories contain a total of 5k+ products. Upon saving the category, the When the category is saved, the I profiled the category save using Xdebug and loaded the data in MacCallGrind and have included some screenshots below demonstrating how much of the processing time was used by the method mentioned above: |
Yeah, AFAIR it worked this way in initial implementation: Maybe, this logic was broken in one of further fixes, like 6c0ac05#diff-33a66f964a57f4cb0f7f47ee7358ed35R106 where some force changes to
Anchor option is responsible for products attachment to category I believe. Another option is responsible for whether URLs of child categories contain parent category |
Any update on this? |
Is there an update on this please, still an issue in 2.1.7 |
If I remember correctly, the fix is in the latest develop branch, but Magento is terribly slow to release it to the general public. We have solved this ourselves in a custom built module for our client.
… On 2 Aug 2017, at 13:15, dan-advantec ***@***.***> wrote:
Is there an update on this please, still an issue in 2.1.7
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub <#7874 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEVa5zq427edlpqtqi5N8Jb4MfJISp3ks5sUFpngaJpZM4LQqWu>.
|
Excellent, thanks @woutersamaey I assume this module isn't released to the general public? |
No, this is closed source, but you can search for the solution in the develop branch or my company can do the work for you and re-package the fix in a stand-alone module if you don’t have the means.
… On 2 Aug 2017, at 13:23, dan-advantec ***@***.***> wrote:
Excellent, thanks @woutersamaey <https://github.com/woutersamaey> I assume this module isn't released to the general public?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#7874 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEVa5wBPK1PMPP_UisHFJCLcp1uwBZrks5sUFwwgaJpZM4LQqWu>.
|
No worries @woutersamaey I'll have a look in the dev branch, but thank you for the offer. |
@dan-advantec @woutersamaey please check https://community.magento.com/t5/Magento-DevBlog/Pull-Requests-for-2-1-x-Patch-Releases/ba-p/65630 out. So, having knowledge where a bugfix resides in |
@dan-advantec you did find any solution ? |
@devonto, thank you for your report. |
Our Upgraded version is 2.2.2 still having the same issue removing one product from the category and then saving category with 4k products taking 5mins to save and some time fails. |
This issue appear when we save product as well. When we un-assigned category with big qty of products and press save product. Because this event also called "catalog_category_save_after". Will be good move logic from this Observer to separate reindex. I had this problem on magento 2.2.3 and with category qty of products > 2-3k |
Stumbled across this thread as a client site is experiencing the same issue. After some more digging, discovered a third party elastic search module was listening for the category save and product save events and rebuilding elastic search which is what was causing the time out. With the event disabled (commented out), saving is back to much more manageable times. Install has 1,700 categories, 50,000 products and 6 store views. Categories save in approx 20 seconds. Worth a quick check to see if you have modules listening to the catalog_category_save event. |
@pixiemediaweb which Elasticsearch module is this? Can you give a bit more detail? |
hi @woutersamaey yes sure - it was the Wyomind module. |
@pixiemediaweb Can you please specify what Wyomind module? Wyomind is a vendor which has multiple modules. |
Hi @quisse - as I said above - its their Elastic Search module |
Oh k, sorry. Forget what I've said. |
no problem |
I feel like this is an issue for me on 2.2.3 and 2.2.5 still. Saving a category can sometimes take like 4 mins. Other categories can save in 10 seconds. Not quite sure why some are effected and not others doesn't seem related to size particularly as root category had 11,000 products and was saving in 10-20 seconds where as a category with 800 products took 4 mins. |
Having the same problem on Magento 2.2.6 ! |
This is really a huge disaster when we are live with lots products and categories and can't unassign products from categories. If anybody have solution, please share. 🙏 |
Using Magento 2.2.5 Same issue. when remove one category from product and then save Product it will throw me error like : Could not save product "55" with position 0 to category 4 {"exception":"[object] (Magento\Framework\Exception\CouldNotSaveException(code: 0): Could not save product "55" with position 0 to category 4 at /home/luxmadein/public_html/vendor/magento/module-catalog/Model/CategoryLinkRepository.php:90, Magento\UrlRewrite\Model\Exception\UrlAlreadyExistsException(code: 0): URL key for specified store already exists. at /home/luxmadein/public_html/vendor/magento/module-url-rewrite/Model/Storage/DbStorage.php:220, Magento\Framework\Exception\AlreadyExistsException(code: 0): URL key for specified store already exists. at /home/luxmadein/public_html/vendor/magento/module-url-rewrite/Model/Storage/DbStorage.php:250, Magento\Framework\DB\Adapter\DuplicateException(code: 1062): SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'hot-chick-6.html-1' for key 'URL_REWRITE_REQUEST_PATH_STORE_ID', query was: INSERT INTO is anybody had found solution ? |
Reopening as I have recently seen this happen on 2.2.7. There are a number of problems involved, one of them being that |
Hi @engcom-backlog-nazar. Thank you for working on this issue.
|
@engcom-backlog-nazar that commit does look promising, at least as far as not regenerating rewrites for products that don't need them regenerated. Thank you for adding that information here. It also looks like it was ported into 2.2.8. I will try generating a patch for this and testing. |
@sjb9774 did you check related commit, with your environment ? is issue still exists ? or we can close this one issue ? |
@engcom-backlog-nazar I just checked with the dev involved with testing that and he has pushed out a patch generated from that commit and has not had any further issues. I would recommend anyone facing this problem upgrade to 2.2.8 or 2.3, or apply a patch generated from that commit. Closing this issue. |
With a catalog of many thousands of products loaded, saving a category causes long execution times and high memory usage, when it isn't necessarily required. In a production environment, the execution fails with
max_execution_time
ormemory_limit
errors.It appears that the system is trying to regenerate all URLs of products contained within the category and subcategories; even if the
url_key
is not modified.Preconditions
max_execution_time
set to 300memory_limit
set to 2GSteps to reproduce
Expected result
You saved the category
should appearActual result
max_execution_time
and/ormemory_limit
A workaround is to modify
/vendor/magento/module-catalog-url-rewrite/Observer/CategoryProcessUrlRewriteSavingObserver.php
temporarily toreturn;
on line 1 of theexecute()
function to prevent URL rewrite changes.A suggested performance improvement would be to check if the
url_key
was modified before running the regeneration of URLs.This, though, cannot be used if the
url_key
is updated. At that point, the process should be handled by some background task.The text was updated successfully, but these errors were encountered: