PHWinfo banniere

Titres
PORTAIL ANNUAIRE ARTICLES COMPARATEUR HÉBERGEURS DEVIS FORUMS RÉDUCTEUR D'URL
Précédent   PHWinfo > Autres forums > Forum Programmation & Conception > comp.lang.php > limiting download size with curl
S'inscrire FAQ Membres Recherche Messages du jour Marquer les forums comme lus
limiting download size with curl

Réponse
 
LinkBack Outils de la discussion
Vieux 26/02/2008, 16h36   #1
shikakaa@gmail.com
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut limiting download size with curl

Hi!

I have a server I'd like to download hundreds of pages from, and only
get the first few kilobytes of every page to save resources on both my
and the remote end. Since the serfver doesn't support HTTP range to
use it in curl like curl_setopt($ch, CURLOPT_RANGE, "0-5000"); I
started looking around and found that using the custom write function
I might be able to do the same.

As I understand it, closing the handle and thus terminating the
transfer from within the callback function is not possible, so I'm a
bit stuck..

How could I achieve this? Could someone paste me a fewliner example?

Thanks in advance
Andrew
  Réponse avec citation
Vieux 26/02/2008, 17h06   #2
Iván Sánchez Ortega
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: limiting download size with curl

shikakaa@gmail.com wrote:

> I have a server I'd like to download hundreds of pages from, and only
> get the first few kilobytes of every page to save resources on both my
> and the remote end.


Have you though using a HTTP HEAD request instead of a HTTP GET request?

Cheers,
--
----------------------------------
Iván Sánchez Ortega -ivansanchez-algarroba-escomposlinux-punto-org-

No es culpable el asesino, sino el asesinado.
  Réponse avec citation
Vieux 26/02/2008, 17h26   #3
Toby A Inkster
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: limiting download size with curl

shikakaa@gmail.com wrote:

> As I understand it, closing the handle and thus terminating the transfer
> from within the callback function is not possible, so I'm a bit stuck..
>
> How could I achieve this?


TCP Sockets. GET requests are not *that* difficult to implement.


--
Toby A Inkster BSc (Hons) ARCS
[Geek of HTML/SQL/Perl/PHP/Python/Apache/Linux]
[OS: Linux 2.6.17.14-mm-desktop-9mdvsmp, up 27 days, 23:43.]

Bottled Water
http://tobyinkster.co.uk/blog/2008/02/18/bottled-water/
  Réponse avec citation
Vieux 26/02/2008, 17h26   #4
shikakaa@gmail.com
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: limiting download size with curl

On Feb 26, 6:06pm, Iván Sánchez Ortega <ivansanchez-...@rroba-
escomposlinux.-.punto.-.org> wrote:
> shika...@gmail.com wrote:
> > I have a server I'd like to download hundreds of pages from, and only
> > get the first few kilobytes of every page to save resources on both my
> > and the remote end.

>
> Have you though using a HTTP HEAD request instead of a HTTP GET request?
>
> Cheers,


As I understand it, the HTTP HEAD request would get me a response that
doesn't have the body of the file in question. The target of my file
downloads are about 20% into body of the file, so this approach
wouldn't work.

Thank you for the suggestion though!
  Réponse avec citation
Vieux 26/02/2008, 19h07   #5
shikakaa@gmail.com
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: limiting download size with curl

On Feb 26, 6:26pm, Toby A Inkster <usenet200...@tobyinkster.co.uk>
wrote:
> shika...@gmail.com wrote:
> > As I understand it, closing the handle and thus terminating the transfer
> > from within the callback function is not possible, so I'm a bit stuck..

>
> > How could I achieve this?

>
> TCP Sockets. GET requests are not *that* difficult to implement.


Could you please show me an example of this? I'm not sure what I'm
supposed to be googling for at this point.
  Réponse avec citation
Vieux 27/02/2008, 01h46   #6
Manuel Lemos
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: limiting download size with curl

Hello,

on 02/26/2008 04:07 PM shikakaa@gmail.com said the following:
> On Feb 26, 6:26 pm, Toby A Inkster <usenet200...@tobyinkster.co.uk>
> wrote:
>> shika...@gmail.com wrote:
>>> As I understand it, closing the handle and thus terminating the transfer
>>> from within the callback function is not possible, so I'm a bit stuck..
>>> How could I achieve this?

>> TCP Sockets. GET requests are not *that* difficult to implement.

>
> Could you please show me an example of this? I'm not sure what I'm
> supposed to be googling for at this point.


You can use this HTTP client class that uses TCP sockets with the PHP
fsockopen and only retrieve the amount of data that you want. It figures
what is the response body and headers so you do not have to write
additional code to parse the headers and skip them:

http://www.phpclasses.org/httpclient


--

Regards,
Manuel Lemos

PHP professionals looking for PHP jobs
http://www.phpclasses.org/professionals/

PHP Classes - Free ready to use OOP components written in PHP
http://www.phpclasses.org/
  Réponse avec citation
Vieux 27/02/2008, 02h06   #7
Jerry Stuckle
Aucun Avatar
 
Messages: n/a
Hébergeur:
Par défaut Re: limiting download size with curl

Manuel Lemos wrote:
> Hello,
>
> on 02/26/2008 04:07 PM shikakaa@gmail.com said the following:
>> On Feb 26, 6:26 pm, Toby A Inkster <usenet200...@tobyinkster.co.uk>
>> wrote:
>>> shika...@gmail.com wrote:
>>>> As I understand it, closing the handle and thus terminating the transfer
>>>> from within the callback function is not possible, so I'm a bit stuck..
>>>> How could I achieve this?
>>> TCP Sockets. GET requests are not *that* difficult to implement.

>> Could you please show me an example of this? I'm not sure what I'm
>> supposed to be googling for at this point.

>
> You can use this HTTP client class that uses TCP sockets with the PHP
> fsockopen and only retrieve the amount of data that you want. It figures
> what is the response body and headers so you do not have to write
> additional code to parse the headers and skip them:
>
> http://www.phpclasses.org/httpclient
>
>


Manuel,

Is the only reason you're on this newsgroup is to promote your junky
classes?


--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex@attglobal.net
==================

  Réponse avec citation
Réponse


Outils de la discussion

Règles de messages
Vous ne pouvez pas créer de nouvelles discussions
Vous ne pouvez pas envoyer des réponses
Vous ne pouvez pas envoyer des pièces jointes
Vous ne pouvez pas modifier vos messages

Les balises BB sont activées : oui
Les smileys sont activés : oui
La balise [IMG] est activée : oui
Le code HTML peut être employé : non
Trackbacks are oui
Pingbacks are oui
Refbacks are oui


Fuseau horaire GMT +1. Il est actuellement 00h27.


Édité par : vBulletin® version 3.7.3
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
Search Engine Friendly URLs by vBSEO 3.2.0 RC5 Tous droits réservés.
Version française #16 par l'association vBulletin francophone
PHWinfo est un site Éducation Sans Frontières ©2000-2008
Ad Management by RedTyger
©Tous droits réservés par les parties respectives
Page generated in 0,13341 seconds with 15 queries