Web Crawling Project

Hamiltonian Consulting develops Custom Web Crawling Application

Web Crawling

For the CityDirect network

Task: A system was developed for web data mining. Users were able to input specialized URLs to guide the crawler, and manually rank data sources for their reliability.

Achievements

  • Gathered large amounts of data (100s of GB) in a relational database (PostgreSQL).
  • Multi-process and multi-threaded application written in Python.
  • Due to code reuse and good practices, delivered system in half the time others had quoted.

137 Responses to “Web Crawling Project”

  1. KENNY says:

    enhancement drugs

    Buy_no prescription…

  2. JOEL says:

    how to get teens off drugs

    Buy_drugs without prescription…

  3. BRADLEY says:

    insulin receptor inhibition

    Buy_without prescription…

  4. RICARDO says:

    sample tlc diets

    Buy_generic drugs…

  5. KURT says:

    can a woman have colorectal cancer

    Buy_drugs without prescription…

  6. BRADLEY says:

    kids weight gain

    Buy_generic drugs…

  7. ALFRED says:

    herbals for pregnancy

    Buy_drugs without prescription…

  8. JEREMY says:

    best@medicare.plan.d.for.nexium.40.mg” rel=”nofollow”>..

    Buygeneric meds…

  9. HARRY says:

    infant@prevacid.dose” rel=”nofollow”>.

    Buygeneric drugs…

Leave a Reply