Jumps are very significant: Model already led to a noticeable increase in Ad conversions: +5% on Insta, +3% on FB in second quarter.
🟢 What's Inside Model?
Three main technical features:
1️⃣ Input data is divided into two groups: sequential features (user action histories, clicks, views, etc.) and non-sequential (location, age, ad properties, etc.). To avoid mixing them all together and blurring signals, a so-called InterFormer with dynamic alternation is used. First, event sequences are processed by a custom transformer block, then a layer combines these outputs with static features through cross-feature interaction blocks, after which the cycle continues at the next level.
2️⃣ Additionally, we need to consider connections between features from the two groups. For this, there is a separate component called Wukong. It consists of stacked factorization machines that search for non-obvious connections between features (why the user behaved one way or another).
3️⃣ For long sequences (i.e., long user histories), a proprietary pyramidal parallel structure is applied. This is needed to avoid the notorious exponential growth in costs as sequence length increases. The entire chain is first split into smaller parts -> they are processed -> the results form the next level of embeddings -> they are split into pieces again and processed -> and so on until everything finally collapses.
As a result, get:
(a) scalability;
(b) the ability to effectively consider all features and their connections;
(c) adequate model behavior on long sequences. And conversion jumps works quite well.
🤖 Data Science, ML & Big Data with @DataXplore