用Ruby做Get网络请求

使用ruby发起网络请求，需要用到'net/http'，下面的程序是获得一个对url请求的
响应

其实最简单的方法是

>>require "open-uri"
>>open("http://www.cnblog.org/blog/atom.xml")

但是，这个方法的缺点是太简单，无法设置超时时间。在超时的情况下，他会无限的请求下去，直到达到了默认的超时时间，这个时间很长

>> open("http://www.cnblog.org/blog/atom.xml")
Errno::ETIMEDOUT: Connection timed out - connect(2)
        from /usr/local/bin/rubyee/lib/ruby/1.8/net/http.rb:560:in `initialize'
        from /usr/local/bin/rubyee/lib/ruby/1.8/net/http.rb:560:in `open'
        from /usr/local/bin/rubyee/lib/ruby/1.8/net/http.rb:560:in `connect'
        from /usr/local/bin/rubyee/lib/ruby/1.8/timeout.rb:53:in `timeout'
        from /usr/local/bin/rubyee/lib/ruby/1.8/timeout.rb:93:in `timeout'
        from /usr/local/bin/rubyee/lib/ruby/1.8/net/http.rb:560:in `connect'
        from /usr/local/bin/rubyee/lib/ruby/1.8/net/http.rb:553:in `do_start'
        from /usr/local/bin/rubyee/lib/ruby/1.8/net/http.rb:542:in `start'
        from /usr/local/bin/rubyee/lib/ruby/1.8/open-uri.rb:242:in `open_http'
        from /usr/local/bin/rubyee/lib/ruby/1.8/open-uri.rb:616:in `buffer_open'
        from /usr/local/bin/rubyee/lib/ruby/1.8/open-uri.rb:164:in `open_loop'
        from /usr/local/bin/rubyee/lib/ruby/1.8/open-uri.rb:162:in `catch'
        from /usr/local/bin/rubyee/lib/ruby/1.8/open-uri.rb:162:in `open_loop'
        from /usr/local/bin/rubyee/lib/ruby/1.8/open-uri.rb:132:in `open_uri'
        from /usr/local/bin/rubyee/lib/ruby/1.8/open-uri.rb:518:in `open'
        from /usr/local/bin/rubyee/lib/ruby/1.8/open-uri.rb:30:in `open'
        from (irb):6>>

为了保险起见，在要考虑超时处理或者其他设定的情况下，还是使用Net::HTTP
除了能设置超时时间之外，还能设置其他的请求参数，例如user-agent

这个user-agent还是很有用的参数，先前在拿163.com做实验的时候，没有设个参数，结果老是重定向，把这个请求当做了手机端的

class HandleGetRequest
  # 对url发起get请求
  require 'net/http'

  def self.get_response(url)
    begin
      url_str = URI.parse(url)
      site = Net::HTTP.new(url_str.host, url_str.port)
      site.open_timeout = 20
      site.read_timeout = 20
      path = url_str.query.blank? ? url_str.path : url_str.path+"?"+url_str.query
      return site.get2(path,{'accept'=>'text/html','user-agent'=>'Mozilla/5.0'})
    rescue Exception => ex
      p ex
    end
  end

end

请求一个正常的网址

>> HandleGetRequest.get_response("http://www.iteye.com/topic/431217")
=> #<Net::HTTPOK 200 OK readbody=true>

如果后面的path为空 注意斜杠

>> HandleGetRequest.get_response("http://www.google.com.hk")
#<ArgumentError: HTTP request path is empty>
=> nil
>> HandleGetRequest.get_response("http://www.google.com.hk/")
=> #<Net::HTTPOK 200 OK readbody=true>

请求一个超时的网址（在我机器上测试时超时的），会在设定的时间到达时抛出异常

>> HandleGetRequest.get_response("http://www.cnblog.org/blog/atom.xml")
#<Timeout::Error: execution expired>
Timeout::Error: execution expired
        from /usr/local/bin/rubyee/lib/ruby/1.8/timeout.rb:60:in `open'
        from /usr/local/bin/rubyee/lib/ruby/1.8/net/http.rb:560:in `connect'
        from /usr/local/bin/rubyee/lib/ruby/1.8/net/http.rb:560:in `connect'
        from /usr/local/bin/rubyee/lib/ruby/1.8/net/http.rb:553:in `do_start'
        from /usr/local/bin/rubyee/lib/ruby/1.8/net/http.rb:542:in `start'
        from /usr/local/bin/rubyee/lib/ruby/1.8/net/http.rb:1035:in `request'
        from /usr/local/bin/rubyee/lib/ruby/1.8/net/http.rb:948:in `get2'
        from /home/chengliwen/chengliwen/deploy/pin-macro-tmp/lib/handle_get_request.rb:30:in `get_response'
        from (irb):1

然后可以根据响应值，去处理response的body了

评论

2 楼 hotsunshine 2010-12-02  

googya 写道

  path = url_str.query.blank? ? url_str.path : url_str.path+"?"+url_str.query  

字符串好像没有blank?这个方法吧（我的是1.9.2）。
如果url 为 www.google.com之类的，那么url_str.path会是空的，会报错的！

这个blank?方法是rails提供的
关于path空，是因为路径斜杠的问题

>> HandleGetRequest.get_response("http://www.google.com.hk")  
#<ArgumentError: HTTP request path is empty>  
=> nil  
>> HandleGetRequest.get_response("http://www.google.com.hk/")  
=> #<Net::HTTPOK 200 OK readbody=true> 

1 楼 googya 2010-12-01  

  path = url_str.query.blank? ? url_str.path : url_str.path+"?"+url_str.query  

字符串好像没有blank?这个方法吧（我的是1.9.2）。
如果url 为 www.google.com之类的，那么url_str.path会是空的，会报错的！