PHP guru needed! file_get_contents()

Status
Not open for further replies.

Jaysin

Banned
Sep 8, 2007
37
0
0
ok i'm writing a board scanner, that goes to boards and opens up all threads and searches for my keywords, works fine... its just i use file_get_contents() to put the whole site into a string for easy searching, i have one problem, its loading images/javascript too which i am assuming is why its slowing it down very bad..

now i will have this set to scan up to around 10 boards, and i tested it on 1 board, works great, except 1 board, and scanning 1 forum category it takes almost 3-5 minutes... i'm sure its because of loading javascript and all that jazz... any ideas how i can make this thing not be so slow?
 


file_get_contents() does this not by itself. the function just opens a file or http connection and "get" the "contents"... nothing more.
 
What tobsn said.

And use (multi)curl if you want it faster. Curl supports gzip encoding etc which should result in faster downloads.
 
ah thanks, any tutorials on curl? i'm new to curl, was to afraid to mess with it so i never did.
 
ah thanks, any tutorials on curl? i'm new to curl, was to afraid to mess with it so i never did.

I use this as a direct replacement for file_get_contents. And yes I ripped if off from somewhere....

PHP:
function file_get_the_contents($url) {
  $ch = curl_init();
  $timeout = 60; // set to zero for no timeout
  curl_setopt ($ch, CURLOPT_URL, $url);
  curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
  curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
  $file_contents = curl_exec($ch);
  curl_close($ch);
  return $file_contents;
}
 
Status
Not open for further replies.