京东爬虫数据Maven环境准备（京东爬虫提交订单）

准备Maven环境

Eclipse集成Maven

新建一个Maven项目

新建一个简单项目

填写Maven信息

项目结构如下

Pom.xml添加如下内容

项目属性

项目属性的内容

<!-- main version setting -->
<spring.version>4.1.9.RELEASE</spring.version>
<validator.version>5.2.4.Final</validator.version>
<mybatis.version>3.2.8</mybatis.version>
<mybatis-spring.version>1.2.3</mybatis-spring.version>
<druid.version>1.0.18</druid.version>
<ehcache.version>2.6.11</ehcache.version>
<ehcache-web.version>2.0.4</ehcache-web.version>
<shiro.version>1.2.3</shiro.version>
<sitemesh.version>2.4.2</sitemesh.version>
<activiti.version>5.21.0</activiti.version>

<!-- tools version setting -->
<slf4j.version>1.7.7</slf4j.version>
<commons-lang3.version>3.3.2</commons-lang3.version>
<commons-io.version>2.4</commons-io.version>
<commons-codec.version>1.9</commons-codec.version>
<!--<commons-codec.version>2.0-SNAPSHOT</commons-codec.version> -->
<commons-fileupload.version>1.3.1</commons-fileupload.version>
<commons-beanutils.version>1.9.1</commons-beanutils.version>
<jackson.version>2.2.3</jackson.version>
<fastjson.version>1.1.40</fastjson.version>
<xstream.version>1.4.7</xstream.version>
<guava.version>17.0</guava.version>
<dozer.version>5.5.1</dozer.version>
<poi.version>3.9</poi.version>
<freemarker.version>2.3.20</freemarker.version>

<!-- jdbc driver setting -->
<mysql.driver.version>5.1.30</mysql.driver.version>
<oracle.driver.version>10.2.0.4.0</oracle.driver.version>
<mssql.driver.version>1.3.1</mssql.driver.version>

<!-- environment setting -->
<jdk.version>1.7</jdk.version>
<tomcat.version>2.2</tomcat.version>
<jetty.version>7.6.14.v20131031</jetty.version>
<webserver.port>8181</webserver.port>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<downloadSources>true</downloadSources>


<!-- 第三方 -->
<alibaba.fastjson.version>1.2.32</alibaba.fastjson.version>
<jackson.mapper.lgpl.version>1.8.1</jackson.mapper.lgpl.version>
<httpclient-version>4.5.2</httpclient-version>

包依赖中配置的内容

<!-- mysql连接包 -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>${mysql.driver.version}</version>
</dependency>
<!-- HttpClient -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>${httpclient-version}</version>
</dependency>
<!-- jsoup -->
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.10.3</version>
</dependency>
<!-- 工具包 -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>${commons-lang3.version}</version>
</dependency>

创建包结构

创建Item类，注意get和set方法

创建HttpClientUtils

编写代码

添加一个测试类

增加Jackson包

准备爬取的网页

测试类添加属性

获取页面数据

解析页面数据

测试运行结果