Scraping Airlines Bots: Insights Obtained Studying Honeypot Data

Airline websites are the victims of unauthorised online travel agencies and aggregators that use armies bots to scrape prices flight information. These so-called Advanced Persistent Bots (APBs) highly sophisticated. On top valuable information taken away, these huge quantities requests consume a very substantial amount resources on airlines' websites. In this work, we propose deceptive approach counter scraping bots. We present platform capable mimicking sites changing at will. provide results case studies performed with it. have lured for almost 2 months, fed them indistinguishable inaccurate Studying collected requests, found behavioural patterns could be used as complementary bot detection. Moreover, based gathered empirical pieces evidence, method investigate claim commonly made proxy services by web millions residential IPs their disposal. Our mathematical models indicate is likely 3 orders magnitude smaller than one claimed. This finding suggests an IP reputation-based blocking strategy effective, contrary what operators think today.