How can I log HTTP::Async communication in Perl? - 【StackMirror】|multithreading|perl|logging

I have a Perl code that use threads and HTTP::Async with multiple outbound IP addresses >>

use threads ( 'yield',
              'exit' => 'threads_only',
              'stack_size' => 2*16384 );
use strict;
use warnings;
 no warnings 'threads';
use threads::shared;
use LWP::UserAgent;
use HTTP::Request;
use HTTP::Async;
...
my $async = HTTP::Async->new( ... );
...
foreach (@list) {
  $thread = threads->create( sub {
    local $SIG{KILL} = sub { threads->exit };
    ...
    $ua->local_address($ip);
    $request->url($url);
    $async->add($request);
    while ($response = $async->wait_for_next_response) {
      ...
    }
  }, $_);
}
...

I need to generate some basic application logs that would include url and outbound IP information.

How can I log HTTP::Async communication?

2012-04-05 22:11
by Ωmega

Well, how would it be logged without threads? That's a start, even if perhaps not initially correct. Then it's just a matter of understanding threads and the impact they have on shared mutable state (including any which may exist within LWP or IO). That is, most of the question/post is irrelevant and can be distilled - NoName 2012-04-05 22:20

You have the requests and the responses. Extract the data you want to log and store it somewhere. Which part is holding you up - brian d foy 2012-04-05 23:20

@briandfoy - For some reason I am afraid some requests go out with incorrect outbound IP address, so logging would help me - Ωmega 2012-04-05 23:59

@brian d foy, The local IP addressed used isn't part of the request or response - ikegami 2012-04-06 01:29

So, the question isn't at all about Perl. It's about what route your networking layer chooses - brian d foy 2012-04-07 02:11

You are not actually using LWP in that code, you are just loading it. Likewise from your code example it's not clear why you are using threads. HTTP::Async takes care of doing multiple HTTP requests "at once".

It doesn't look like HTTP::Async has any built-in debug mechanisms, but it'd be relatively straightforward to add a 'debug' flag in that module and some warn statements at the appropriate places if you need help seeing what it's doing. You could also add the debug code in your own code that's using HTTP::Async.

2012-04-05 22:34
by Ask Bjørn Hansen

Thanks. Based on you answer I edited the question subject, so now it says "How can I log HTTP::Async communication in Perl..?" - Ωmega 2012-04-05 22:40