window下安装爬虫框架scrapy

在window环境下安装scrapy

   1】进入到python的安装目录下：C:\Python27\Scripts

   2】进入到命令行模式，执行如下命令;
      pip install wheel

   3】下载附件中的文件，将两个可执行的文件点击执行

   4】将lxml-3.6.4-cp27-cp27m-win_amd64.whl 文件复制到C:\Python27\Scripts目录下，执行 pip install lxml-3.6.4-cp27-cp27m-win_amd64.whl

   5】上述步骤完成后执行 pip install scrapy

   6】安装后执行scrapy命令，如果出现如下提示：
   
      

[C:\Python27\Scripts]$ scrapy
Traceback (most recent call last):
  File "c:\python27\lib\runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "c:\python27\lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\Python27\Scripts\scrapy.exe\__main__.py", line 5, in <module>
  File "c:\python27\lib\site-packages\scrapy\cmdline.py", line 9, in <module>
    from scrapy.crawler import CrawlerProcess
  File "c:\python27\lib\site-packages\scrapy\crawler.py", line 7, in <module>
    from twisted.internet import reactor, defer
  File "c:\python27\lib\site-packages\twisted\internet\reactor.py", line 38, in <module>
    from twisted.internet import default
  File "c:\python27\lib\site-packages\twisted\internet\default.py", line 56, in <module>
    install = _getInstallFunction(platform)
  File "c:\python27\lib\site-packages\twisted\internet\default.py", line 50, in _getInstallFunction
    from twisted.internet.selectreactor import install
  File "c:\python27\lib\site-packages\twisted\internet\selectreactor.py", line 18, in <module>
    from twisted.internet import posixbase
  File "c:\python27\lib\site-packages\twisted\internet\posixbase.py", line 18, in <module>
    from twisted.internet import error, udp, tcp
  File "c:\python27\lib\site-packages\twisted\internet\tcp.py", line 28, in <module>
    from twisted.internet._newtls import (
  File "c:\python27\lib\site-packages\twisted\internet\_newtls.py", line 21, in <module>
    from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
  File "c:\python27\lib\site-packages\twisted\protocols\tls.py", line 63, in <module>
    from twisted.internet._sslverify import _setAcceptableProtocols
  File "c:\python27\lib\site-packages\twisted\internet\_sslverify.py", line 38, in <module>
    TLSVersion.TLSv1_1: SSL.OP_NO_TLSv1_1,
AttributeError: 'module' object has no attribute 'OP_NO_TLSv1_1'

   7】请执行pip install Twisted==13.1.0

   8】完成后在执行scrapy出现如下提示即安装成功：

     

[C:\Python27\Scripts]$ scrapy
Scrapy 1.5.0 - no active project

Usage:
  scrapy <command> [options] [args]

Available commands:
  bench         Run quick benchmark test
  fetch         Fetch a URL using the Scrapy downloader
  genspider     Generate new spider using pre-defined templates
  runspider     Run a self-contained spider (without creating a project)
  settings      Get settings values
  shell         Interactive scraping console
  startproject  Create new project
  version       Print Scrapy version
  view          Open URL in browser, as seen by Scrapy

  [ more ]      More commands available when run from project directory

Use "scrapy <command> -h" to see more info about a command