CentOS7下安装geowave+hbase+geoserver的实现

概述

GeoWave是由国家地理空间情报局(NGA)与RadiantBlue和Booz Allen Hamilton合作技术开发的。GeoWave是一套类库，利用分布式键值存储的可伸缩性，从而有效的存储、检索和分析大量的地理数据集。目前是通过提供插件连接GeoTools和PDAL

Accumulo基础数据存储。GeoWave的主要目标是成为流行的地理空间项目之间的桥梁,领域的分布式键值存储和分布式处理框架。地理空间操作往往是后期工作，或不能与许多这样的存储和计算能力融合。通过GeoWave让他们可以支持大众的应用提供一流的工具。

GeoWave的地理空间支持是建立在GeoTools扩展模型之上的,所以插件本身支持GeoServer,uDig和任何其他GeoTools兼容的项目,以及可以摄取GeoTools兼容的数据源。GeoWave是基于Accumulo开发实现而产生的。

HBase – Hadoop Database，是一个高可靠性、高性能、面向列、可伸缩的分布式存储系统，利用HBase技术可在廉价PC Server上搭建起大规模结构化存储集群。

GeoServer 是 OpenGIS Web 服务器规范的 J2EE 实现，利用 GeoServer 可以方便的发布地图数据，允许用户对特征数据进行更新、删除、插入操作，通过 GeoServer 可以比较容易的在用户之间迅速共享空间地理信息。

基础信息

参考url:

https://www.linkedin.com/pulse/testing-big-spatial-data-software-hadoop-hbase-geowave-alvaro-huarte/

https://blog.csdn.net/sinat_35582093/article/details/80182019

安装geoserver2.13.2参考

https://blog.csdn.net/hsg77/article/details/81910007

安装hbase参考

https://blog.csdn.net/hsg77/article/details/81037726

安装的环境：

master192.168.10.180namenode,datanode,hmaster,hregionserver

slave1 192.168.10.181namenode,datanode,hmaster,hregionserver

slave2 192.168.10.182datanode,hregionserver

geoserver192.168.10.190tomcat geoserver geowave

geoserver服务器hosts具体内容：

127.0.0.1 core localhost.localdomain localhost

192.168.10.190 geoserver

192.168.10.180 master

192.168.10.181 slave1

192.168.10.182 slave2

hdfs webpage:http://192.168.10.180:50070http://192.168.10.181:50070

hbase webpage:http://192.168.10.180:16010/master-statushttp://192.168.10.181:16010/master-status

tomcat webpage:http://192.168.10.190:8080/

geoserver webpageurl:

安装geowave0.9.7

下载离线包：geowave-geoserver-0.9.7-apache.jar

http://s3.amazonaws.com/geowave-rpms/release-jars/JAR/geowave-geoserver-0.9.7-apache.jar

更多版本地址：

http://locationtech.github.io/geowave/packages.html

选择Release JARS有一个列表下载

将它拷到 /usr/tomcat/webapps/geoserver/WEB-INF/lib下

此时启动geoserver, 打开浏览器进入 192.168.10.190:8080/geoserver/web 登录geoserver。默认帐号admin，密码geoserver。

在左侧data栏目下store, 点击add new store可以看到geowave插件已经加载了

使用hbase，需要把geowave-hbase-0.9.7-apache.jar拷到hbase的lib里面

具体位置为：/home/hadoop3/app/hbase/lib

三台服务器都要拷贝(与hbase有关) master,slave1,slave2

数据准备情况：

在geoserver192.168.10.190服务器上执行下面的代码：

[root@geoserver geoserver]# pwd

/usr/geoserver

#拷贝文件到这个目录下

geowave-tools-0.9.7-apache.jarne_50m_admin_0_countries.shp

[root@geoserver geoserver]# ls -ls

总用量 395328

90944 -rw-r--r--. 1 root root 93122657 7月 23 21:53 geoserver.war

302124 -rw-r--r--. 1 root root 309371727 8月 23 16:23 geowave-tools-0.9.7-apache.jar

20 -rw-r--r--. 1 root root 18008 6月 19 2017 GPL.txt

80 -rw-r--r--. 1 root root 81585 2月 20 2018 LICENSE.txt

4 -rw-r--r--. 1 root root 5 8月 23 16:22 ne_50m_admin_0_countries.cpg

536 -rw-r--r--. 1 root root 546979 8月 23 16:22 ne_50m_admin_0_countries.dbf

4 -rw-r--r--. 1 root root 143 8月 23 16:22 ne_50m_admin_0_countries.prj

24 -rw-r--r--. 1 root root 23632 8月 23 16:22 ne_50m_admin_0_countries.README.html

4 -rw-r--r--. 1 root root 2396 8月 23 16:22 ne_50m_admin_0_countries.sbn

4 -rw-r--r--. 1 root root 268 8月 23 16:22 ne_50m_admin_0_countries.sbx

1576 -rw-r--r--. 1 root root 1612740 8月 23 16:22 ne_50m_admin_0_countries.shp

4 -rw-r--r--. 1 root root 2028 8月 23 16:22 ne_50m_admin_0_countries.shx

4 -rw-r--r--. 1 root root 7 8月 23 16:22 ne_50m_admin_0_countries.VERSION.txt

0 drwxr-xr-x. 2 root root 25 7月 23 22:00 target

[root@geoserver geoserver]#

利用geowave将shp导入hbase并将该store添加到geoserver中

查看geowave版本

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar il.nga.giat.geowave.core.cli.GeoWaveMain –version

修改zookeeper默认地址

查看已有数据源列表

java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config list

找到其中的zookeeper的key设置值

default.opts.zookeeper为config list中看到具体key为准

java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config set default.opts.zookeeper master:2181

建立空间索引

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config addindex -t spatial eea-spindex --partitionStrategy

在hbase里面建立geowave的eea-store

注意最后的端口，如果你修改过hbase的zookeeper记得修改

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config addstore eea-store --gwNamespace geowave.eea -t hbase --zookeeper 192.168.10.180:2181

将本地shp导入刚刚建立的eea-store中

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain ingest localtogw -f geotools-vector ./ne_50m_admin_0_countries.shp eea-store eea-spindex

配置geoserver连接

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config geoserver -ws geowave -u admin -p geoserver http://192.168.10.190:8080/geoserver

利用geowave给geoserver添加layer

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain gs addlayer eea-store

查看web页面情况

http://192.168.10.190:8080/geoserver

数据存储：

geowave eea-store-vector GeoWave Datastore – HBASE

需要手动添加图层：

ne_50m_admin_0_countries geowave:ne_50m_admin_0_countries eea-store-vector

具体过程如下所示：

导入./ne_50m_admin_0_countries.shp到hbase中并发布到geoserver中

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain --version

{build.os=Linux

build.jvm.vendor=Oracle Corporation

build.jvm.version=1.8.0_131-b11

build.maven.version=3.2.2

build.os.arch=amd64

build.os.version=4.9.27-14.31.amzn1.x86_64

build.timestamp=2018-03-28T16:45:31+0000

build.user=root

project.branch=UNKNOWN

project.build.args=-Dvendor.version=apache

project.scm.revision=dbac917aff060ee206bd7b7f5dec3fa3db6ef422

project.version=0.9.7}

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config addindex -t spatial eea-spindex --partitionStrategy ROUND_ROBIN

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config addstore eea-store --gwNamespace geowave.eea -t hbase --zookeeper 192.168.10.180:2181

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain ingest localtogw -f geotools-vector ./ne_50m_admin_0_countries.shp eea-store eea-spindex

23 八月 16:38:32 WARN [util.NativeCodeLoader] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

23 八月 16:38:32 INFO [zookeeper.RecoverableZooKeeper] - Process identifier=hconnection-0xa2431d0 connecting to ZooKeeper ensemble=192.168.10.180:2181

八月 23, 2018 4:38:34 下午 org.geotools.image.ImageWorker <clinit>

信息: Warp/affine reduction enabled: true

八月 23, 2018 4:38:34 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 4:38:34 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 4:38:34 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

八月 23, 2018 4:38:34 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionProvider', but ApplicationContext is unset.

八月 23, 2018 4:38:34 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

八月 23, 2018 4:38:34 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 4:38:34 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 4:38:34 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

八月 23, 2018 4:38:34 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionProvider', but ApplicationContext is unset.

八月 23, 2018 4:38:34 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config geoserver -ws geowave -u admin -p geoserver http://192.168.10.190:8080/geoserver

geoserver.url=http://192.168.10.190:8080/geoserver

geoserver.pass=ENC{68336f69636339454a43743136667a34644b456b4b673d3d}

geoserver.user=admin

geoserver.workspace=geowave

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain gs addlayer eea-store

23 八月 16:40:54 WARN [util.NativeCodeLoader] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

23 八月 16:40:54 INFO [zookeeper.RecoverableZooKeeper] - Process identifier=hconnection-0x738dc9b connecting to ZooKeeper ensemble=192.168.10.180:2181

八月 23, 2018 4:40:55 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 4:40:55 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 4:40:55 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

八月 23, 2018 4:40:55 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionProvider', but ApplicationContext is unset.

八月 23, 2018 4:40:55 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

八月 23, 2018 4:40:55 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 4:40:55 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 4:40:55 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

八月 23, 2018 4:40:55 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionProvider', but ApplicationContext is unset.

八月 23, 2018 4:40:55 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

Add GeoServer layer for 'eea-store: OK : {

"description": "Successfully added:",

"layers": [ {

"id": "ne_50m_admin_0_countries",

"type": "vector"

}]

}

[root@geoserver geoserver]#

#导入另一个shp文件到hbase并发布到geoserver中

./guizhou_xzq.shp

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config addindex -t spatial gzxzq-spindex --partitionStrategy ROUND_ROBIN

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config addstore gzxzq-store --gwNamespace geowave.gzxzq -t hbase --zookeeper 192.168.10.180:2181

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain ingest localtogw -f geotools-vector ./guizhou_xzq.shp gzxzq-store gzxzq-spindex

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config geoserver -ws geowave -u admin -p geoserver http://192.168.10.190:8080/geoserver

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain gs addlayer gzxzq-store

具体过程如下：

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config addindex -t spatial gzxzq-spindex --partitionStrategy ROUND_ROBIN

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config addstore gzxzq-store --gwNamespace geowave.gzxzq -t hbase --zookeeper 192.168.10.180:2181

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain ingest localtogw -f geotools-vector ./guizhou_xzq.shp gzxzq-store gzxzq-spindex

23 八月 18:04:52 WARN [util.NativeCodeLoader] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

23 八月 18:04:53 INFO [zookeeper.RecoverableZooKeeper] - Process identifier=hconnection-0xa2431d0 connecting to ZooKeeper ensemble=192.168.10.180:2181

八月 23, 2018 6:04:55 下午 org.geotools.image.ImageWorker <clinit>

信息: Warp/affine reduction enabled: true

八月 23, 2018 6:04:56 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 6:04:56 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 6:04:56 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

八月 23, 2018 6:04:56 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionProvider', but ApplicationContext is unset.

八月 23, 2018 6:04:56 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

八月 23, 2018 6:04:56 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 6:04:56 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 6:04:56 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

八月 23, 2018 6:04:56 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionProvider', but ApplicationContext is unset.

八月 23, 2018 6:04:56 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

23 八月 18:05:00 INFO [client.HBaseAdmin] - Created geowave.gzxzq_GEOWAVE_METADATA

23 八月 18:05:05 INFO [client.HBaseAdmin] - Created geowave.gzxzq_SPATIAL_IDX

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config geoserver -ws geowave -u admin -p geoserver http://192.168.10.190:8080/geoserver

geoserver.pass=ENC{68336f69636339454a43743136667a34644b456b4b673d3d}

geoserver.workspace=geowave

geoserver.url=http://192.168.10.190:8080/geoserver

geoserver.user=admin

[root@geoserver geoserver]# java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain gs addlayer gzxzq-store

23 八月 18:07:22 WARN [util.NativeCodeLoader] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

23 八月 18:07:22 INFO [zookeeper.RecoverableZooKeeper] - Process identifier=hconnection-0x738dc9b connecting to ZooKeeper ensemble=192.168.10.180:2181

八月 23, 2018 6:07:24 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 6:07:24 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 6:07:24 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

八月 23, 2018 6:07:24 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionProvider', but ApplicationContext is unset.

八月 23, 2018 6:07:24 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

八月 23, 2018 6:07:24 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 6:07:24 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'GeoServerResourceLoader', but ApplicationContext is unset.

八月 23, 2018 6:07:24 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

八月 23, 2018 6:07:24 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionProvider', but ApplicationContext is unset.

八月 23, 2018 6:07:24 下午 org.geoserver.platform.GeoServerExtensions checkContext

警告: Extension lookup 'ExtensionFilter', but ApplicationContext is unset.

Add GeoServer layer for 'gzxzq-store: OK : {

"description": "Successfully added:",

"layers": [ {

"id": "guizhou_xzq",

"type": "vector"

}]

}

[root@geoserver geoserver]#

添加2000坐标系35度带的shp文件方法

xzq35.shp

java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config addindex -t spatial xzq35-spindex --partitionStrategy ROUND_ROBIN --crs EPSG:4523

java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config addstore xzq35-store --gwNamespace geowave.xzq35 -t hbase --zookeeper master:2181,slave1:2181,slave2:2181

java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain ingest localtogw -f geotools-vector ./xzq35.shp xzq35-store xzq35-spindex

java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain config geoserver -ws geowave -u admin -p geoserver http://192.168.145.190:8080/geoserver

java -cp ./geowave-tools-0.9.7-apache.jar mil.nga.giat.geowave.core.cli.GeoWaveMain gs addlayer xzq35-store

25 九月 17:27:43 INFO [client.HBaseAdmin] - Created geowave.xzq35_GEOWAVE_METADATA

25 九月 17:27:48 INFO [client.HBaseAdmin] - Created geowave.xzq35_SPATIAL_IDX_4523