Archive.org website scraper

[teomax]

Hi
I need to download content for few expired domains from archive.org, does anyone have scrapper for archive.org for sale? I dont want to do it by hand and I tried a several downloaders, but they dont work that way, unfortunately.

 

[Jon]

I heard ALL of the blackhats do it by hand because it builds character... you should try it.

 

[RuDeDoGg]

what do u want to download, specific content or the entire page?

 

[chatmasta]

I heard ALL of the blackhats do it by hand because it builds character... you should try it.

hahahaha!

I assume this is like Eli's desert scraping post though? When his blog comes back online, you could check the comments there...or the squirt forum if you're a member.....or PM eli.

 

[ebiz]

Doing it by hand?

* Goes around to think about another service to engage a bunch of indian labourers*

 

[rogerd]

The CPAN Search Site - search.cpan.org

If you know Perl, I've used this and it works really well for quickly restoring a site.

 

[plepco]

The CPAN Search Site - search.cpan.org

If you know Perl, I've used this and it works really well for quickly restoring a site.

Damn skippy. One of the lesser-known tools out there. I just started learning Perl and this is an example of WHY: there are a LOT of powerful scripts people have written.

+rep for the link

 

[georgiecas1y]

I used warrick for this purpose: Warrick - Recover Your Lost Website

It's a linux command line tool, written in perl as well i think

 

[tobsn]

if someone would read my weblog... i did such a scraper...
results are on vserv.info - its build after eli's real desert scraping article.

ESS: The Real Desert Scraping | A Blackhat’s Diary

there are also some basic code snippets how i started the thing...
it scrapes archive.org and cleans the content.


damn why is nobody reading my blog? ...