How do people deal, in general, with inconsistent data in Yves?

UKGT7RC7P

How do people deal, in general, with inconsistent data in Yves?

UKGT7RC7P

or if people encounter this issue at all?

UKGT7RC7P

get so many NULL errors which after digging involve missing or inconsistent data

UK5DS29L2

missing comparing to what?

UKHD8KTMF

never had those, sry

UKGT7RC7P

never had null errors? 🙂

UKHD8KTMF

never had inconsistent data - whatever that means

UKHD8KTMF

I might have removed something and then there was an error in twig or something but that is kinda expected

UKHD8KTMF

but I might be missing what you mean

UKGT7RC7P

for example, right now I have a product which is being pulled from the product storage and crashing because the abstrat storage data for that product isn't there

UKGT7RC7P

and it's calling json_decode on NULL

UKHD8KTMF

hmm, ok this is a concrete problem

UKGT7RC7P

but one way or another related to data inconsistency

UKGT7RC7P

we have previously thought about a health check or similar which would ensure that the data in Redis / ES is sane

UKHD8KTMF

yes, now i get your point - sync to the storage fails for some reason and you end up in broken state

UKHD8KTMF

I really do not see any solution but having monitoring set up and detecting when product page gets and error

UM4BZSK7T

As far as I know we never had that in Yves as well but we use a heavy custom hack.

UKGT7RC7P

so right now I could override the Spryker class to deal with this eventuality somehow - even if it's an explicit log message and an exception

UKGT7RC7P

@UM4BZSK7T what is the heavy custom hack? 🙂

UM4BZSK7T

I guess there is nothing left how spryker uses data in Yves as we created templates that putout VueJS stuff. And I can't recall any Redis data being out of sync.

UM4BZSK7T

We have problems with the event system not syncing. But never json decode on NULL

UKGT7RC7P

yeah, this is just an instance of this type of issue

UKGT7RC7P

previously also had issues with prices etc (on another project)

UM4BZSK7T

Prices for example are found by a call to our ERP and not to Yves afaik

UKHD8KTMF

I also have most experience with heavily customized shops

UNBSW8S8K

Non-existing data in storages is one scenario, wait until you will get to the point where you will reach race-conditions under high load, then it will get fun.

UKGT7RC7P

yeah -- not sure if theres anything to handle f.e. prices being published some time after product data

UKGT7RC7P

not sure though 😄

UNBSW8S8K

When we went for consultation @ spryker, their senior architect stated there is nothing we can do about that. So 🤷‍♂️

Andriy Netseplyayev

@UKGT7RC7P but your issue seems like missing event triggers in pub and sync. How did you get to the situation when you have concrete product with missing abstract?

UKGT7RC7P

yeah, this is what I'm investigating now 😉