Awesome Open Source
Awesome Open Source

Outbrain Click Prediction challenge solution

Overview:

The part of the solution is a combination of 5 models:

  • SVM and FTRL on basic features:
    • event features: user id, document id, platform id, day, hour and geo
    • ad features: ad document id, campaign, advertizer id
  • XGB and ET on MTV (Mean Target Value) features:
    • all categorical features that previous model used
    • document features like publisher, source, top category, topic and entity
    • interaction between these featuers
    • also, the document similarity features: the cosine between the ad doc and the page with the ad
  • FFM with the following features:
    • all categorical features from the above, except document similarity, categories, topics and entities
    • XGB leaves from the previous step (see slide 9 from this presentation for the description of the idea)
  • The models are combined with an XGB model (rank:pairwise objective)

To get the 13th positions, models from diaman should also be added

Files description

  • 0_prepare_splits.py splits the training dataset into two folds
  • 1_svm_data.py prepares the data for SVM and FTRL
  • 1_train_ftrl.py and 1_train_svm.py train models on data from 1_svm_data.py
  • 2_extract_leaked_docs.py and 2_leak_features.py extract the leak
  • 3_doc_similarity_features.py calculates TF-IDF similarity between the document user on and the ad document
  • 4_categorical_data_join.py and 4_categorical_data_unwrap_columnwise.py prepare data for MTV features calculation
  • 4_mean_target_value.py calculates MTV for all features from categorical_features.txt
  • 5_best_mtv_features_xgb.py builds an XBG on a small part of data and selects best features to be used on for XGB and ET
  • 5_mtv_et.py trains ET model on MTV features
  • 5_mtv_xgb.py trains XGB model on MTV features and creates leaf featurse to be used in FFM
  • 6_1_generate_ffm_data.py creates the input file to be read by ffmlib
  • 6_2_split_ffm_to_subfolds.py splits each fold into two subfolds (can't use the original folds because the leaf features are not transferable between folds)
  • 6_3_run_ffm.sh runs libffm for training FFM models
  • 6_4_put_ffm_subfolds_together.py puts FFM predictions from each fold/subfold together
  • 7_ensemble_data_prep.py puts all the features and model predictions together for ensembling
  • 7_ensemble_xgb.py traings the second level XGB model on top of all these features

The files should be run in the above order

Diaman's features should be included into 7_ensemble_data_prep.py - and the rest can stay unchanged.


Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
python (55,543
kaggle (111
xgboost (63