`

httpclient 4.0 使用 - 访问开心网的各种组件 例子 转

ApachePHPthreadJ# 
阅读更多

在网上查了好长时间，发现httpclient的例子都是讲httpclient 3.X的。httpclient 4.0的例子简直是凤毛麟角。只好自己上官网仔细了查了下文档和官网示例，受益匪浅啊！
    由于前段时间对“开心网”的“开心大亨”很感兴趣，但是每隔一段时间就需要看一下最新的价格变化，很是麻烦，由于我就想起用httpclient写个程序，自动查看最新的价格，并和前次的价格做比较，好让我可以选择利润最大的进行投资。
   
    程序用到的包如下，为了附件的瘦身，请大家自己下载啊，附件中是没有这些lib的啊。
    commons-collections-3.2.1.jar
    commons-configuration-1.6.jar
    commons-io-1.4.jar
    commons-lang-2.4.jar
    commons-logging-1.1.1.jar
    httpclient-4.0.jar
    httpcore-4.0.1.jar

    由于只是例子，而且httpclient4.0的结构变化很大，自己对其理解也不是很深，此程序只是个入门程序，如果有高手在的话，希望大家可以提出宝贵的意见。
    程序写的不是很美观，但意在让大家明白如何简单的使用httpclient4.0，所以没有对程序做任何的重构的整理，望大家多多谅解啊。

    程序代码如下：
   
    主程序： KaiXin001.java

Java代码 复制代码
  1. /*  
  2.  * 2009/12/21  
  3.  * Author:  Yuan Hongzhi   
  4.  */  
  5. import java.io.File;   
  6. import java.io.InputStream;   
  7. import java.util.ArrayList;   
  8. import java.util.List;   
  9.   
  10. import org.apache.commons.io.FileUtils;   
  11. import org.apache.http.HttpEntity;   
  12. import org.apache.http.HttpResponse;   
  13. import org.apache.http.NameValuePair;   
  14. import org.apache.http.client.entity.UrlEncodedFormEntity;   
  15. import org.apache.http.client.methods.HttpGet;   
  16. import org.apache.http.client.methods.HttpPost;   
  17. import org.apache.http.cookie.Cookie;   
  18. import org.apache.http.entity.BufferedHttpEntity;   
  19. import org.apache.http.impl.client.DefaultHttpClient;   
  20. import org.apache.http.message.BasicNameValuePair;   
  21. import org.apache.http.protocol.HTTP;   
  22. import org.apache.http.util.EntityUtils;   
  23.   
  24. public class KaiXin001 {   
  25.   
  26.   public InputStream getResourceAsStream(String filename) throws Exception {   
  27.     ClassLoader classLoader = Thread.currentThread().getContextClassLoader();   
  28.   
  29.     InputStream in = null;   
  30.     if (classLoader != null) {   
  31.       in = classLoader.getResourceAsStream(filename);   
  32.     }   
  33.     if (in == null) {   
  34.       in = ClassLoader.getSystemResourceAsStream(filename);   
  35.     }   
  36.     if (in == null) {   
  37.       throw new Exception("Can't find resource file: " + filename);   
  38.     } else {   
  39.       return in;   
  40.     }   
  41.   }   
  42.   
  43.   public void writeToFile(String file, HttpEntity entity) throws Exception {   
  44.     writeToFile(file, EntityUtils.toString(entity));   
  45.   }   
  46.   
  47.   public void writeToFile(String file, String data) throws Exception {   
  48.     FileUtils.writeStringToFile(new File(file), data, "UTF-8");   
  49.   }   
  50.   
  51.   public String getContent(HttpEntity entity) throws Exception {   
  52.     if (entity != null) {   
  53.       entity = new BufferedHttpEntity(entity);   
  54.       long len = entity.getContentLength();   
  55.       System.out.println("Length: " + len);   
  56.       System.out.println("====================\r\n");   
  57.       return EntityUtils.toString(entity, "UTF-8");   
  58.     } else {   
  59.       System.out.println("entity is null.");   
  60.       return null;   
  61.     }   
  62.   }   
  63.   
  64.   public void setCookie(DefaultHttpClient httpclient, List<Cookie> cookies) {   
  65.     if (cookies.isEmpty()) {   
  66.       System.out.println("Cookie is empty.");   
  67.       return;   
  68.     } else {   
  69.       for (int i = 0; i < cookies.size(); i++) {   
  70.         System.out.println((i + 1) + " - " + cookies.get(i).toString());   
  71.         httpclient.getCookieStore().addCookie(cookies.get(i));   
  72.       }   
  73.       System.out.println();   
  74.     }   
  75.   }   
  76.   
  77.   // "开心网其它组件URL如下，大家可以添加上自己喜欢的组件URL。"   
  78.   // "http://www.kaixin001.com/!slave/index.php", "朋友买卖"   
  79.   // "http://www.kaixin001.com/!parking/index.php", "争车位"   
  80.   // "http://www.kaixin001.com/!house/index.php?_lgmode=pri", "买房子"   
  81.   // "http://www.kaixin001.com/!house/index.php?_lgmode=pri&t=49"   
  82.   // "http://www.kaixin001.com/!house/garden/index.php","花园"   
  83.   // "http://www.kaixin001.com/!rich/market.php", "超级大亨"   
  84.   public String enterComponentContent(String url, String componentName,   
  85.       DefaultHttpClient httpclient, List<Cookie> cookies,   
  86.       HttpResponse response, HttpEntity entity) throws Exception {   
  87.     System.out.println("--- Enter: " + componentName + " ---");   
  88.     System.out.println("--- Url:   " + url + " ---");   
  89.     setCookie(httpclient, cookies);   
  90.     HttpGet httpget = new HttpGet(url);   
  91.     response = httpclient.execute(httpget);   
  92.   
  93.     entity = response.getEntity();   
  94.     return getContent(entity);   
  95.   }   
  96.   
  97.   public void showResponseStatus(HttpResponse response) {   
  98.     // System.out.println(response.getProtocolVersion());   
  99.     // System.out.println(response.getStatusLine().getStatusCode());   
  100.     // System.out.println(response.getStatusLine().getReasonPhrase());   
  101.     System.out.println(response.getStatusLine().toString());   
  102.     System.out.println("-------------------------\r\n");   
  103.   }   
  104.   
  105.   public static void main(String[] args) throws Exception {   
  106.     KaiXin001 kx = new KaiXin001();   
  107.     DefaultHttpClient httpclient = new DefaultHttpClient();   
  108.   
  109.     HttpPost httpost = new HttpPost("http://www.kaixin001.com/login/login.php");   
  110.     List<NameValuePair> qparams = new ArrayList<NameValuePair>();   
  111.     qparams.add(new BasicNameValuePair("email", "email"));   
  112.     qparams.add(new BasicNameValuePair("password", "password"));   
  113.   
  114.     httpost.setEntity(new UrlEncodedFormEntity(qparams, HTTP.UTF_8));   
  115.     HttpResponse response = httpclient.execute(httpost);   
  116.     kx.showResponseStatus(response);   
  117.   
  118.     // HttpResponse response = httpclient.execute(httpget);   
  119.     HttpEntity entity = response.getEntity();   
  120.     kx.getContent(entity);   
  121.   
  122.     // Login   
  123.     List<Cookie> cookies = httpclient.getCookieStore().getCookies();   
  124.     System.out.println("Post logon cookies:");   
  125.     kx.setCookie(httpclient, cookies);   
  126.   
  127.     // Redirect to home page   
  128.     String homepage = "http://www.kaixin001.com/home/";   
  129.     String content = null;   
  130.     content = kx.enterComponentContent(homepage, "Home page", httpclient,   
  131.         cookies, response, entity);   
  132.   
  133.     // Component   
  134.     String componet = "http://www.kaixin001.com/!rich/market.php";   
  135.     content = kx.enterComponentContent(componet, "Component", httpclient,   
  136.         cookies, response, entity);   
  137.   
  138.     // --------------------------------------------   
  139.     kx.writeToFile("c:/kaixin.html", content);   
  140.   
  141.     // When HttpClient instance is no longer needed,   
  142.     // shut down the connection manager to ensure   
  143.     // immediate deallocation of all system resources   
  144.     httpclient.getConnectionManager().shutdown();   
  145.   }   
  146. }  
/*
 * 2009/12/21
 * Author:  Yuan Hongzhi 
 */
import java.io.File;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;

import org.apache.commons.io.FileUtils;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.cookie.Cookie;
import org.apache.http.entity.BufferedHttpEntity;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.protocol.HTTP;
import org.apache.http.util.EntityUtils;

public class KaiXin001 {

  public InputStream getResourceAsStream(String filename) throws Exception {
    ClassLoader classLoader = Thread.currentThread().getContextClassLoader();

    InputStream in = null;
    if (classLoader != null) {
      in = classLoader.getResourceAsStream(filename);
    }
    if (in == null) {
      in = ClassLoader.getSystemResourceAsStream(filename);
    }
    if (in == null) {
      throw new Exception("Can't find resource file: " + filename);
    } else {
      return in;
    }
  }

  public void writeToFile(String file, HttpEntity entity) throws Exception {
    writeToFile(file, EntityUtils.toString(entity));
  }

  public void writeToFile(String file, String data) throws Exception {
    FileUtils.writeStringToFile(new File(file), data, "UTF-8");
  }

  public String getContent(HttpEntity entity) throws Exception {
    if (entity != null) {
      entity = new BufferedHttpEntity(entity);
      long len = entity.getContentLength();
      System.out.println("Length: " + len);
      System.out.println("====================\r\n");
      return EntityUtils.toString(entity, "UTF-8");
    } else {
      System.out.println("entity is null.");
      return null;
    }
  }

  public void setCookie(DefaultHttpClient httpclient, List<Cookie> cookies) {
    if (cookies.isEmpty()) {
      System.out.println("Cookie is empty.");
      return;
    } else {
      for (int i = 0; i < cookies.size(); i++) {
        System.out.println((i + 1) + " - " + cookies.get(i).toString());
        httpclient.getCookieStore().addCookie(cookies.get(i));
      }
      System.out.println();
    }
  }

  // "开心网其它组件URL如下，大家可以添加上自己喜欢的组件URL。"
  // "http://www.kaixin001.com/!slave/index.php", "朋友买卖"
  // "http://www.kaixin001.com/!parking/index.php", "争车位"
  // "http://www.kaixin001.com/!house/index.php?_lgmode=pri", "买房子"
  // "http://www.kaixin001.com/!house/index.php?_lgmode=pri&t=49"
  // "http://www.kaixin001.com/!house/garden/index.php","花园"
  // "http://www.kaixin001.com/!rich/market.php", "超级大亨"
  public String enterComponentContent(String url, String componentName,
      DefaultHttpClient httpclient, List<Cookie> cookies,
      HttpResponse response, HttpEntity entity) throws Exception {
    System.out.println("--- Enter: " + componentName + " ---");
    System.out.println("--- Url:   " + url + " ---");
    setCookie(httpclient, cookies);
    HttpGet httpget = new HttpGet(url);
    response = httpclient.execute(httpget);

    entity = response.getEntity();
    return getContent(entity);
  }

  public void showResponseStatus(HttpResponse response) {
    // System.out.println(response.getProtocolVersion());
    // System.out.println(response.getStatusLine().getStatusCode());
    // System.out.println(response.getStatusLine().getReasonPhrase());
    System.out.println(response.getStatusLine().toString());
    System.out.println("-------------------------\r\n");
  }

  public static void main(String[] args) throws Exception {
    KaiXin001 kx = new KaiXin001();
    DefaultHttpClient httpclient = new DefaultHttpClient();

    HttpPost httpost = new HttpPost("http://www.kaixin001.com/login/login.php");
    List<NameValuePair> qparams = new ArrayList<NameValuePair>();
    qparams.add(new BasicNameValuePair("email", "email"));
    qparams.add(new BasicNameValuePair("password", "password"));

    httpost.setEntity(new UrlEncodedFormEntity(qparams, HTTP.UTF_8));
    HttpResponse response = httpclient.execute(httpost);
    kx.showResponseStatus(response);

    // HttpResponse response = httpclient.execute(httpget);
    HttpEntity entity = response.getEntity();
    kx.getContent(entity);

    // Login
    List<Cookie> cookies = httpclient.getCookieStore().getCookies();
    System.out.println("Post logon cookies:");
    kx.setCookie(httpclient, cookies);

    // Redirect to home page
    String homepage = "http://www.kaixin001.com/home/";
    String content = null;
    content = kx.enterComponentContent(homepage, "Home page", httpclient,
        cookies, response, entity);

    // Component
    String componet = "http://www.kaixin001.com/!rich/market.php";
    content = kx.enterComponentContent(componet, "Component", httpclient,
        cookies, response, entity);

    // --------------------------------------------
    kx.writeToFile("c:/kaixin.html", content);

    // When HttpClient instance is no longer needed,
    // shut down the connection manager to ensure
    // immediate deallocation of all system resources
    httpclient.getConnectionManager().shutdown();
  }
}




    访问开心大亨组件的程序：KaiXin_Rich.java

Java代码 复制代码
  1. /*  
  2.  * 2009/12/21  
  3.  * Author:  Yuan Hongzhi   
  4.  */  
  5. import java.awt.Toolkit;   
  6. import java.text.DecimalFormat;   
  7. import java.util.ArrayList;   
  8. import java.util.List;   
  9. import java.util.regex.Matcher;   
  10. import java.util.regex.Pattern;   
  11.   
  12. import org.apache.commons.collections.keyvalue.DefaultKeyValue;   
  13. import org.apache.commons.configuration.PropertiesConfiguration;   
  14. import org.apache.http.HttpEntity;   
  15. import org.apache.http.HttpResponse;   
  16. import org.apache.http.NameValuePair;   
  17. import org.apache.http.client.entity.UrlEncodedFormEntity;   
  18. import org.apache.http.client.methods.HttpPost;   
  19. import org.apache.http.cookie.Cookie;   
  20. import org.apache.http.impl.client.DefaultHttpClient;   
  21. import org.apache.http.message.BasicNameValuePair;   
  22. import org.apache.http.protocol.HTTP;   
  23.   
  24. public class KaiXin_Rich extends KaiXin001 {   
  25.   
  26.   public static List<DefaultKeyValue> curList = new ArrayList<DefaultKeyValue>();   
  27.   
  28.   public static List<DefaultKeyValue> preList = new ArrayList<DefaultKeyValue>();   
  29.   
  30.   public void copyList(List<DefaultKeyValue> src, List<DefaultKeyValue> des) {   
  31.     for (DefaultKeyValue dkv : src) {   
  32.       DefaultKeyValue kv = new DefaultKeyValue(dkv.getKey(), dkv.getValue());   
  33.       des.add(kv);   
  34.     }   
  35.   }   
  36.   
  37.   public Double subNumber(Object string) {   
  38.     String rex = "([\\d\\.]*)";   
  39.     Pattern pattern = Pattern.compile(rex);   
  40.     Matcher match = pattern.matcher(string.toString());   
  41.     if (match.find()) {   
  42.       return Double.valueOf(match.group());   
  43.     } else  
  44.       return -1.0;   
  45.   }   
  46.   
  47.   public String extractNoneAscii(Object string) {   
  48.     String rex = "([^\\x00-\\xff]+)";   
  49.     Pattern pattern = Pattern.compile(rex);   
  50.     Matcher match = pattern.matcher(string.toString());   
  51.     if (match.find()) {   
  52.       return match.group();   
  53.     } else  
  54.       return "";   
  55.   }   
  56.   
  57.   public static void main(String[] args) throws Exception {   
  58.     KaiXin_Rich kx = new KaiXin_Rich();   
  59.   
  60.     PropertiesConfiguration config = new PropertiesConfiguration();   
  61.     config.load(kx.getResourceAsStream("properties/kaixin001.properties"),   
  62.         "UTF-8");   
  63.   
  64.     DefaultHttpClient httpclient = new DefaultHttpClient();   
  65.   
  66.     HttpPost httpost = new HttpPost("http://www.kaixin001.com/login/login.php");   
  67.     List<NameValuePair> qparams = new ArrayList<NameValuePair>();   
  68.     qparams.add(new BasicNameValuePair("email", config.getString("email")));   
  69.     qparams   
  70.         .add(new BasicNameValuePair("password", config.getString("password")));   
  71.   
  72.     httpost.setEntity(new UrlEncodedFormEntity(qparams, HTTP.UTF_8));   
  73.     HttpResponse response = httpclient.execute(httpost);   
  74.     kx.showResponseStatus(response);   
  75.   
  76.     // HttpResponse response = httpclient.execute(httpget);   
  77.     HttpEntity entity = response.getEntity();   
  78.     kx.getContent(entity);   
  79.   
  80.     // Login   
  81.     List<Cookie> cookies = httpclient.getCookieStore().getCookies();   
  82.     System.out.println("Post logon cookies:");   
  83.     kx.setCookie(httpclient, cookies);   
  84.   
  85.     // Redirect to home page   
  86.     String homepage = "http://www.kaixin001.com/home/";   
  87.     String content = null;   
  88.     content = kx.enterComponentContent(homepage, "Home page", httpclient,   
  89.         cookies, response, entity);   
  90.   
  91.     int loopTime = config.getInt("times");   
  92.     long sleepTime = config.getLong("period") * 60;   
  93.     for (int j = 0; j < loopTime; j++) {   
  94.       // Component   
  95.       String componet = config.getString("url");   
  96.       content = kx.enterComponentContent(componet, "Component", httpclient,   
  97.           cookies, response, entity);   
  98.   
  99.       int beginIndex = content.indexOf(config.getString("startRex"));   
  100.       int endIndex = content.lastIndexOf(config.getString("endRex"));   
  101.       content = content.substring(beginIndex, endIndex);   
  102.       // --------------------------------------------   
  103.       String rex = config.getString("displayKeyRex");   
  104.       Pattern pattern = Pattern.compile(rex, Pattern.MULTILINE);   
  105.       Matcher match = pattern.matcher(content);   
  106.       List<String> keys = new ArrayList<String>();   
  107.       String matched = null;   
  108.       while (match.find()) {   
  109.         matched = match.group();   
  110.         keys.add(matched);   
  111.       }   
  112.   
  113.       rex = config.getString("displayValueRex");   
  114.       pattern = Pattern.compile(rex, Pattern.MULTILINE);   
  115.       match = pattern.matcher(content);   
  116.       List<String> values = new ArrayList<String>();   
  117.       while (match.find()) {   
  118.         matched = match.group();   
  119.         values.add(matched);   
  120.       }   
  121.   
  122.       DefaultKeyValue kv = null;   
  123.       for (int i = 0; i < keys.size(); i++) {   
  124.         kv = new DefaultKeyValue(keys.get(i), values.get(i));   
  125.         curList.add(kv);   
  126.       }   
  127.   
  128.       String rtnString = "";   
  129.       String eachLine = null;   
  130.       for (DefaultKeyValue dkv : curList) {   
  131.         eachLine = dkv.getKey() + "\t:" + dkv.getValue();   
  132.         rtnString += eachLine;   
  133.         rtnString += "\r\n";   
  134.         System.out.println(eachLine);   
  135.       }   
  136.   
  137.       System.out.println("\r\n===== Changed =====\r\n");   
  138.       String allPriceString = "===== Changed =====\r\n\r\n";   
  139.       StringBuffer price = null;   
  140.       double changed = 0;   
  141.       boolean beepFlag = false;   
  142.       for (int i = 0; i < curList.size() && curList.size() == preList.size(); i++) {   
  143.         double curMoney = kx.subNumber(curList.get(i).getValue());   
  144.         double preMoney = kx.subNumber(preList.get(i).getValue());   
  145.         if ((curMoney - preMoney) > 1E-5) {   
  146.           changed = curMoney - preMoney;   
  147.   
  148.           DecimalFormat format = new DecimalFormat("###.##");   
  149.           String number = format.format(changed);   
  150.   
  151.           price = new StringBuffer();   
  152.           price.append(curList.get(i).getKey());   
  153.           price.append("\t: ");   
  154.           price.append(preList.get(i).getValue());   
  155.           price.append(" -> ");   
  156.           price.append(curList.get(i).getValue());   
  157.           price.append("\t");   
  158.           price.append(" Changed: ");   
  159.           price.append(number);   
  160.           price.append(kx.extractNoneAscii(curList.get(i).getValue()));   
  161.           System.out.println(price);   
  162.           allPriceString = allPriceString + price.toString() + "\r\n";   
  163.           beepFlag = true;   
  164.         }   
  165.       }   
  166.       if (beepFlag) {   
  167.         Toolkit.getDefaultToolkit().beep();   
  168.       }   
  169.       preList.clear();   
  170.   
  171.       // System.out.println(content);   
  172.       String dateTime = DateUtils   
  173.           .getSystemDateTime(DateTimePattern.DATE_TIME_LONG_UNFORMAT);   
  174.       kx.writeToFile("c:/kaixin001/price_" + dateTime + ".txt", rtnString   
  175.           + "\r\n" + allPriceString);   
  176.       // kx.writeToFile("c:/kaixin.html", content);   
  177.   
  178.       kx.copyList(curList, preList);   
  179.       curList.clear();   
  180.       Thread.sleep(sleepTime * 1000);   
  181.     }   
  182.   
  183.     // When HttpClient instance is no longer needed,   
  184.     // shut down the connection manager to ensure   
  185.     // immediate deallocation of all system resources   
  186.     httpclient.getConnectionManager().shutdown();   
  187.   }   
  188. }  
/*
 * 2009/12/21
 * Author:  Yuan Hongzhi 
 */
import java.awt.Toolkit;
import java.text.DecimalFormat;
import java.util.ArrayList;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.apache.commons.collections.keyvalue.DefaultKeyValue;
import org.apache.commons.configuration.PropertiesConfiguration;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.cookie.Cookie;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.protocol.HTTP;

public class KaiXin_Rich extends KaiXin001 {

  public static List<DefaultKeyValue> curList = new ArrayList<DefaultKeyValue>();

  public static List<DefaultKeyValue> preList = new ArrayList<DefaultKeyValue>();

  public void copyList(List<DefaultKeyValue> src, List<DefaultKeyValue> des) {
    for (DefaultKeyValue dkv : src) {
      DefaultKeyValue kv = new DefaultKeyValue(dkv.getKey(), dkv.getValue());
      des.add(kv);
    }
  }

  public Double subNumber(Object string) {
    String rex = "([\\d\\.]*)";
    Pattern pattern = Pattern.compile(rex);
    Matcher match = pattern.matcher(string.toString());
    if (match.find()) {
      return Double.valueOf(match.group());
    } else
      return -1.0;
  }

  public String extractNoneAscii(Object string) {
    String rex = "([^\\x00-\\xff]+)";
    Pattern pattern = Pattern.compile(rex);
    Matcher match = pattern.matcher(string.toString());
    if (match.find()) {
      return match.group();
    } else
      return "";
  }

  public static void main(String[] args) throws Exception {
    KaiXin_Rich kx = new KaiXin_Rich();

    PropertiesConfiguration config = new PropertiesConfiguration();
    config.load(kx.getResourceAsStream("properties/kaixin001.properties"),
        "UTF-8");

    DefaultHttpClient httpclient = new DefaultHttpClient();

    HttpPost httpost = new HttpPost("http://www.kaixin001.com/login/login.php");
    List<NameValuePair> qparams = new ArrayList<NameValuePair>();
    qparams.add(new BasicNameValuePair("email", config.getString("email")));
    qparams
        .add(new BasicNameValuePair("password", config.getString("password")));

    httpost.setEntity(new UrlEncodedFormEntity(qparams, HTTP.UTF_8));
    HttpResponse response = httpclient.execute(httpost);
    kx.showResponseStatus(response);

    // HttpResponse response = httpclient.execute(httpget);
    HttpEntity entity = response.getEntity();
    kx.getContent(entity);

    // Login
    List<Cookie> cookies = httpclient.getCookieStore().getCookies();
    System.out.println("Post logon cookies:");
    kx.setCookie(httpclient, cookies);

    // Redirect to home page
    String homepage = "http://www.kaixin001.com/home/";
    String content = null;
    content = kx.enterComponentContent(homepage, "Home page", httpclient,
        cookies, response, entity);

    int loopTime = config.getInt("times");
    long sleepTime = config.getLong("period") * 60;
    for (int j = 0; j < loopTime; j++) {
      // Component
      String componet = config.getString("url");
      content = kx.enterComponentContent(componet, "Component", httpclient,
          cookies, response, entity);

      int beginIndex = content.indexOf(config.getString("startRex"));
      int endIndex = content.lastIndexOf(config.getString("endRex"));
      content = content.substring(beginIndex, endIndex);
      // --------------------------------------------
      String rex = config.getString("displayKeyRex");
      Pattern pattern = Pattern.compile(rex, Pattern.MULTILINE);
      Matcher match = pattern.matcher(content);
      List<String> keys = new ArrayList<String>();
      String matched = null;
      while (match.find()) {
        matched = match.group();
        keys.add(matched);
      }

      rex = config.getString("displayValueRex");
      pattern = Pattern.compile(rex, Pattern.MULTILINE);
      match = pattern.matcher(content);
      List<String> values = new ArrayList<String>();
      while (match.find()) {
        matched = match.group();
        values.add(matched);
      }

      DefaultKeyValue kv = null;
      for (int i = 0; i < keys.size(); i++) {
        kv = new DefaultKeyValue(keys.get(i), values.get(i));
        curList.add(kv);
      }

      String rtnString = "";
      String eachLine = null;
      for (DefaultKeyValue dkv : curList) {
        eachLine = dkv.getKey() + "\t:" + dkv.getValue();
        rtnString += eachLine;
        rtnString += "\r\n";
        System.out.println(eachLine);
      }

      System.out.println("\r\n===== Changed =====\r\n");
      String allPriceString = "===== Changed =====\r\n\r\n";
      StringBuffer price = null;
      double changed = 0;
      boolean beepFlag = false;
      for (int i = 0; i < curList.size() && curList.size() == preList.size(); i++) {
        double curMoney = kx.subNumber(curList.get(i).getValue());
        double preMoney = kx.subNumber(preList.get(i).getValue());
        if ((curMoney - preMoney) > 1E-5) {
          changed = curMoney - preMoney;

          DecimalFormat format = new DecimalFormat("###.##");
          String number = format.format(changed);

          price = new StringBuffer();
          price.append(curList.get(i).getKey());
          price.append("\t: ");
          price.append(preList.get(i).getValue());
          price.append(" -> ");
          price.append(curList.get(i).getValue());
          price.append("\t");
          price.append(" Changed: ");
          price.append(number);
          price.append(kx.extractNoneAscii(curList.get(i).getValue()));
          System.out.println(price);
          allPriceString = allPriceString + price.toString() + "\r\n";
          beepFlag = true;
        }
      }
      if (beepFlag) {
        Toolkit.getDefaultToolkit().beep();
      }
      preList.clear();

      // System.out.println(content);
      String dateTime = DateUtils
          .getSystemDateTime(DateTimePattern.DATE_TIME_LONG_UNFORMAT);
      kx.writeToFile("c:/kaixin001/price_" + dateTime + ".txt", rtnString
          + "\r\n" + allPriceString);
      // kx.writeToFile("c:/kaixin.html", content);

      kx.copyList(curList, preList);
      curList.clear();
      Thread.sleep(sleepTime * 1000);
    }

    // When HttpClient instance is no longer needed,
    // shut down the connection manager to ensure
    // immediate deallocation of all system resources
    httpclient.getConnectionManager().shutdown();
  }
}

 

 

个人签名

-------------------------------------

 

图盾 淘宝保护 保护图片 图片防盗

分享到：
| jsp p3p跨域设置cookie - 翻过这座山，就 ...
评论
发表评论

您还没有登录,请您登录后再发表评论

相关推荐

    原创： httpclient 4.0 使用 - 访问开心网的各种组件 例子

    本文将详细讲解如何使用 HttpClient 4.0 访问开心网的各种组件，并通过具体实例进行深入解析。 一、HttpClient 4.0 概述 HttpClient 4.0 提供了对 HTTP 协议的全面支持，包括 GET、POST、PUT、DELETE 等方法，以及...

    httpclient-4.0-beta2.jar

    httpclient-4.0-beta2.jarhttpclient-4.0-beta2.jarhttpclient-4.0-beta2.jarhttpclient-4.0-beta2.jarhttpclient-4.0-beta2.jarhttpclient-4.0-beta2.jarhttpclient-4.0-beta2.jarhttpclient-4.0-beta2.jar

    httpclient-4.0-beta1.jar

    httpclient-4.0-beta1.jar

    用httpclient-4.0-alpha2 打造基于http协议的网站分析器

    在实际开发中，我们需要以下步骤来使用HttpClient-4.0-alpha2： 1. 创建HttpClient实例：根据项目需求，可以设置连接池、超时时间、重试策略等。 ```java CloseableHttpClient httpClient = HttpClients.create...

    httpclient4.0-jar

    1. `httpclient-4.3.2.jar`：这是HttpClient的主要库，包含了HTTP客户端的核心类和接口，如`HttpClient`、`HttpGet`、`HttpPost`等。 2. `httpcore-nio-4.3.2.jar`：这个库提供了非阻塞I/O的支持，是HttpClient实现...

    httpclient4.0 使用帮助、例子

    本篇文章将详细介绍HTTPClient 4.0的使用方法，包括其核心概念、基本操作和示例代码。 一、核心概念 1. HttpClient实例：HttpClient对象是执行HTTP请求的核心，负责建立连接、发送请求和接收响应。通过`...

    httpclient4.0

    二、HttpClient 4.0 的关键组件 1. `HttpClient` 类：作为客户端的核心，它负责创建和管理 HTTP 请求，以及处理服务器的响应。 2. `HttpRequestBase` 和 `HttpResponse`：分别表示 HTTP 请求和响应的基础抽象类，...

    httpclient-4.5.6-API文档-中文版.zip

    赠送jar包：httpclient-4.5.6.jar； 赠送原API文档：httpclient-4.5.6-javadoc.jar； 赠送源代码：httpclient-4.5.6-sources.jar； 赠送Maven依赖信息文件：httpclient-4.5.6.pom； 包含翻译后的API文档：httpclient...

    httpclient-4.0.jar

    httpclient-4.0.jar， httpclient-4.0.jar， httpclient-4.0.jar

    httpcore-4.2.4，httpclient-4.2.5，httpclient-cache-4.2.5，httpmime-4.2.5的jar包下载

    这里提到的四个jar包——httpcore-4.2.4，httpclient-4.2.5，httpclient-cache-4.2.5，httpmime-4.2.5，都是Apache HttpClient库的不同组件，用于支持HTTP通信和相关功能。 **httpcore-4.2.4.jar** 是HTTP Core模块...

    httpclient-4.5.13-API文档-中文版.zip

    赠送jar包：httpclient-4.5.13.jar； 赠送原API文档：httpclient-4.5.13-javadoc.jar； 赠送源代码：httpclient-4.5.13-sources.jar； 赠送Maven依赖信息文件：httpclient-4.5.13.pom； 包含翻译后的API文档：...

    httpclient-4.5.6-API文档-中英对照版.zip

    赠送jar包：httpclient-4.5.6.jar； 赠送原API文档：httpclient-4.5.6-javadoc.jar； 赠送源代码：httpclient-4.5.6-sources.jar； 赠送Maven依赖信息文件：httpclient-4.5.6.pom； 包含翻译后的API文档：httpclient...

    httpclient-4.1-alpha1.jar

    httpclient-4.1-alpha1.jar httpclient-4.1-alpha1.jar httpclient-4.1-alpha1.jar httpclient-4.1-alpha1.jar httpclient-4.1-alpha1.jar

    httpclient-4.4.1-API文档-中文版.zip

    赠送jar包：httpclient-4.4.1.jar； 赠送原API文档：httpclient-4.4.1-javadoc.jar； 赠送源代码：httpclient-4.4.1-sources.jar； 赠送Maven依赖信息文件：httpclient-4.4.1.pom； 包含翻译后的API文档：httpclient...

    Android升级后旧版本的httpclient4.0

    Android升级后旧版本的httpclient4.0 apache-mime4j-0.6 commons-codec-1.4 commons-logging-1.1.1 httpclient-4.0.1 httpcore-4.0.1 httpmime-4.0.1

    httpclient-4.5.13-API文档-中英对照版.zip

    赠送jar包：httpclient-4.5.13.jar； 赠送原API文档：httpclient-4.5.13-javadoc.jar； 赠送源代码：httpclient-4.5.13-sources.jar； 赠送Maven依赖信息文件：httpclient-4.5.13.pom； 包含翻译后的API文档：...

    httpclient-4.5.12-API文档-中英对照版.zip

    赠送jar包：httpclient-4.5.12.jar； 赠送原API文档：httpclient-4.5.12-javadoc.jar； 赠送源代码：httpclient-4.5.12-sources.jar； 赠送Maven依赖信息文件：httpclient-4.5.12.pom； 包含翻译后的API文档：...

    httpclient-4.5.5-API文档-中文版.zip

    赠送jar包：httpclient-4.5.5.jar； 赠送原API文档：httpclient-4.5.5-javadoc.jar； 赠送源代码：httpclient-4.5.5-sources.jar； 包含翻译后的API文档：httpclient-4.5.5-javadoc-API文档-中文(简体)版.zip ...

Magicbox
Global site tag (gtag.js) - Google Analytics