AnjLab Harvester

Harvester is a system designed for collecting and storing selling offers available on online marketplaces, such as Amazon, Half, etc (for now 22 marketplaces are supported). The system gathers products and offers data in endless cycle, collected information can be used for further analysis.

Harvester is multi-component, distributed and scalable software system. It includes central control component (Tracker) and set of agents (scanners) which execute set of tasks (jobs). The system can be easy scaled up with ability to increase \ decrease number of running agents and re-allocate tasks among them. This approach allows monitoring the system performance and eliminating 'bottlenecks'. Moreover, Harvester is self-regulated system and can dynamically change one's configuration relying on defined 'targets' and current situation.

Harvester is specially designed for using benefits of virtualization. The system can use resources of virtual machines or resources provided by 'computing clouds' services, for instance, Amazon Elastic Computing Cloud. Architecture of Harvester can be adopted for gathering any structured information in the Internet.

Company: BetterWorldBooks, IN, USA.

Business: collecting and re-selling books online through19 online marketplaces, including Amazon.com, Ebay, and their own online e-commerce site, betterworld.com

Story: since 2008, BetterWorldBooks uses Harvester to collect book information available on internet markets.

Harvester Administrative web interface.

Harvester Job Monitor.