Calling Developers!
We are reenergizing our code contribution process! Learn More

What are the Slack Archives?

It’s a history of our time together in the Slack Community! There’s a ton of knowledge in here, so feel free to search through the archives for a possible answer to your question.

Because this space is not active, you won’t be able to create a new post or comment here. If you have a question or want to start a discussion about something, head over to our categories and pick one to post in! You can always refer back to a post from Slack Archives if needed; just copy the link to use it as a reference..

What is the relationship with `publish:trigger-events` and `sync:data` ? If I run `publish:trigger-e

Options
U03T08LE3CP
U03T08LE3CP Posts: 185 🧑🏻‍🚀 - Cadet

What is the relationship with publish:trigger-events and sync:data ? If I run publish:trigger-events when do I need to run sync:data?

«1

Comments

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    publish:trigger-events triggers all propel entity events (as far as I understand, probably the update event for all entities) which leads to the first step of publish / synchronize being executed: the denormalized data is written to the _storage and _search tables in the database. This then automatically triggers the events that lead to the sync part of the publish / sync: writing all the data from _storage tables to redis and all the data from _search tables to elasticsearch. the sync:data command only does the last part, transfering the data from _storage and _search tables to redis and elasticsearch (while _storage and _search are only naming conventions. I think they can be named differently, and also there are at least 1 or 2 tables with _search suffix that do something different)

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    so if you only want to sync for example from spy_product_concrete_page_search to elasticsearch, sync:data (-r product_concrete I think also works) would be sufficient. if spy_product_concrete_page_search does not contain the latest data, you need publish:trigger-events (-r product_concrete) to publish the data from spy_product to spy_product_concrete_page_search

  • U03T08LE3CP
    U03T08LE3CP Posts: 185 🧑🏻‍🚀 - Cadet
    Options

    so if I do say, manipulations to image sets in the database directly, should it suffice to publish:trigger-events -r product_abstract_image?

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    yes, it should 👍 (I think 😛 )

  • U03T08LE3CP
    U03T08LE3CP Posts: 185 🧑🏻‍🚀 - Cadet
    Options

    I'm starting to get really hesitant to make any SQL changes since I cannot seem to get the indices update cleanly..

  • U03T08LE3CP
    U03T08LE3CP Posts: 185 🧑🏻‍🚀 - Cadet
    Options

    now the imageSets value in glue abstract_products is empty and I don't know why

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    hm ok. So you can go step by step from the database side, and look at the spy_product_abstract_image_storage table and see if it contains the correct data

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    and if yes, the next step would be to look into the storage (probably redis). you can get the redis key for this from the spy_product_abstract_image_storage table

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    or one question: are you querying one specific product or a list of products? if it is a list it might be _search table and elasticsearch instead

  • U03T08LE3CP
    U03T08LE3CP Posts: 185 🧑🏻‍🚀 - Cadet
    Options

    queyring ong product

  • U03T08LE3CP
    U03T08LE3CP Posts: 185 🧑🏻‍🚀 - Cadet
    edited May 2023
    Options

    /abstract-products/001_4059729207241?include=concrete-products,abstract-product-image-sets,concrete-product-prices,concrete-product-availabilities,category-nodes,concrete-product-image-sets

  • U03T08LE3CP
    U03T08LE3CP Posts: 185 🧑🏻‍🚀 - Cadet
    Options

    something like that

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    then the data should come from the redis storage which is (or should be) synced from the spy_product_abstract_image_storage table. I'm not 100% sure about the glue part (because I did not use glue myself, yet): You included the image sets, maybe there is also another include needed for the actual image data?

  • U03T08LE3CP
    U03T08LE3CP Posts: 185 🧑🏻‍🚀 - Cadet
    Options

    I know this is right it just doesn't reflect the db changes

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    while there is only one storage table for abstract images so probably they are combined during publish

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    then you should look at the spy_product_abstract_image_storage table and search for the product to see if this is correct

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    you might also want to look at rabbitmq, if there are any messages on one of the publish.error queues. If so, you can look at the messages in the rabbitmq UI and see if they contain an error message (if something is going wrong during publish)

  • U03T08LE3CP
    U03T08LE3CP Posts: 185 🧑🏻‍🚀 - Cadet
    Options

    storage looks right

  • U03T08LE3CP
    U03T08LE3CP Posts: 185 🧑🏻‍🚀 - Cadet
    Options

    error queues are empty

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    then the problem is probably that the data is not updated to redis (so the sync:data part). Depending on your infrastructure you might be able to use some client to connect to it and look at the values. But there is also a UI in Zed to access it under maintenance -> storage I think

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    you can use the key for the data set from the spy_product_abstract_image_storage table and search for it in the storage UI

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    (in Zed it must be an exact match of the key I think)

  • U03T08LE3CP
    U03T08LE3CP Posts: 185 🧑🏻‍🚀 - Cadet
    Options

    yeah it's redis 🙂

  • U03T08LE3CP
    U03T08LE3CP Posts: 185 🧑🏻‍🚀 - Cadet
    Options

    but I have tinkered so that I have cli access to it as well

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    ok, I'm not very familiar with the cli (other then deleting everything 😄 ), but you should be able to search for the key there quite easily

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    I think its just GET ... 😛

  • U03T08LE3CP
    U03T08LE3CP Posts: 185 🧑🏻‍🚀 - Cadet
    Options

    the root cause might be that I didn't quite get how the product-image connection works

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    in glue?

  • U03T08LE3CP
    U03T08LE3CP Posts: 185 🧑🏻‍🚀 - Cadet
    Options

    in the database

  • U01BZ7Q3XRV
    U01BZ7Q3XRV Posts: 148 🧑🏻‍🚀 - Cadet
    Options

    ah ok