Scrapebox API

Status
Not open for further replies.
acidie

acidie

A=A
May 27, 2008
1,063
33
0
I'm writing a program that will allow programmatic access to Scrapebox and want to get peoples thoughts on it.

It's in a very Alpha state at the moment but you can see a basic example of its ability in the video below;

[ame=http://www.youtube.com/watch?v=jvoKklH5Au4]YouTube - scrapebox api test[/ame]


A quick explanation of what is going on in the video. When I click the first button it enters the text "powered by wordpress" in the footprint textbox, it then loads the file "USCities.txt" in to the keyword list.

When I click the second button it loads the keyword scraper and scrapes the keyword "monkey foot".

What I'm plaining is the program acting as an API by accepting commands (via JSON over HTTP) and then issuing them to SB, when SB has finished it would then report back to a defined URL with the results.

The actions could also be chained so commands could be issued like;

scrape keywords -> scrape search engines -> scrape back links -> analyze blogs

And they would be performed in sequence either reporting after each step is complete or at the end of the chain.

Thoughts, ideas, abuse?
 


I've been flat out busy the last couple of days, but I got a bit of free time ATM so I'm going to work on releasing a version in next couple of days so people can test for it out for themselves.

It will only demonstrate one function, you'll be able to run the keyword scraper with the keyword of your choice, but it will show the power behind SB API.

Thanks for the comments and interest. I'll post any updates to this thread so it easy to track.
 
Subscribed, this sounds fucking awesome. I was just thinking about this the other day and wishing there was something like it out there, instead of having to load it up on my home comp.
 
phrench said:
Are you planning to make other/all SB functions available via the API?
Click to expand...

Yes I plan on exposing all the functions of Scrapebox including Addons via the API. Some sections of Scrapebox such as options and setting I don't plan on being accessible, mainly because these would seldom need to be changed, if at all.
 
Just adding some finishing touches, but in the meantime;

[ame="http://www.youtube.com/watch?v=o1NEfwiLQrI"]YouTube - scrapebox api beta[/ame]
 
The beta is ready. You can grab it here Scrapebox API Bridge Beta

A few things to note;

So I don't have to type Scrapebox API Bridge over and over I'm going to refer to it as SBAPIB.

While I have tested the programs on various machines, they still might crash, do weird shit, etc. They should work without issue, but this is a beta so some issue are expected.

The zip file contains 2 files, SendJSON and Scrapebox API Bridge. While you don't need to run SendJSON for SBAPIB to work, I include it so you don't have to mess around with ports and Apache, lighttpd, etc.

Also SBAPIB needs Scrapebox to be running and visible (not hidden to the taskbar) for it to find it.

Thats great acidie, but how do I use this fucker?

Ok so now the fun begins, run Scrapebox then run SBAPIB and SendJSON. SBAPIB should report it has found Scrapebox, if it reports it can't find Scrapebox close SBAPIB, make sure Scrapebox is running and visible then run SBAPIB again.

After you have done that you can click the "Send" button on SendJSON.

You will see text appear in both SendJSON and SBAPIB, this is those two programs talking to each other using JSON over HTTP.

In SBAPIB you will see the JSON you sent it from SendJSON.

In SendJSON you will a list of responses from SBAPIB like the following;

Code: 
{"SessionToken":"123","Response":"Starting"}
{"SessionToken":"123","Response":"[WindowOpen] Finished"}
{"SessionToken":"123","Response":"Harverster.KeywordScaprer Starting"}
{"SessionToken":"123","Response":"[Keywords] Finished"}
{"SessionToken":"123","Response":"[Scrape] Finished"}
{"SessionToken":"123","Response":"[RemoveDuplicates] Finished"}
{"SessionToken":"123","Response":"[AddMainList] Finished"}
{"SessionToken":"123","Response":"Harverster.KeywordScaprer Complete"}
{"SessionToken":"123","Response":"Complete"}

Ok cool but you said I don't have to use the SendJSON program

No you don't, you can use anything that can communicate via JSON over HTTP.

For example you could send commands to SBAPIB using PHP, Python, Ruby, Perl, etc.

To do his you need to call 'http://localhost:1337/' with a POST body of the JSON.

For the beta the IP and Port is fixed and can not be changed, but the port shouldn't clash with other programs.

If you want to receive postback messages, like SendJSON does, you need to have something (like apache, lighttpd, etc) listening on port 1338.

Changing commands

SendJSON provides a basic set commands for SBAPIB but there is one that I have left out which you can use.

The command is called 'TransferKeywordList' which will transfer the scraped keywords in to the keyword list. You can use this commend to loop the scrape like in the example JSON below.

Code: 
{
    "Commands": {
        "Harverster.KeywordScaprer": [
            {
                "Keywords": [
                    "wickedfire"
                ]
            },
            "Scrape",
            "RemoveDuplicates",
            "TransferKeywordList",
            "Scrape",
            "RemoveDuplicates",
            "AddMainList"
        ]
    },
    "Authentication": "81f0bcd8-e632-46f9-bb17-f174f3732762",
    "SessionToken": "87894654"
}

I have a problem xxx

When I was testing the beta 90% of the problems were caused by firewalls. Check that first then if your still having issues report the problem in this thread with a basic description of the what happened (or a crashdump is even better) and what type of computer you were running it on, for example Win 2008 64bit.

Known issues

On some systems the first time you run command through SBAPIB it finishes without any problems. But if you run commands (the same or new ones) a second time SBAPIB will stall half way through and never complete.

I know this is a problem and I'm working on fixing it.

Last but not least

I would greatly appreciate any feedback, positive or negative, about SBAPIB. So if you could leave feedback that would be great, thanks.
 
  • Like
Reactions: gutterseo
I just realized I forgot to mention minimum running requirements.

Both SBAPIB and SendJSON require the .NET framework version 3.5 or higher.
 
Status
Not open for further replies.
Top Bottom