Merge branch 'relations' into travis
commit
863d6d4dc0
6
Makefile
6
Makefile
|
@ -34,11 +34,13 @@ clean:
|
||||||
rm -f imposm3
|
rm -f imposm3
|
||||||
(cd test && make clean)
|
(cd test && make clean)
|
||||||
|
|
||||||
test: test-unit test-system
|
test: imposm3
|
||||||
|
$(GO) test ./... -i
|
||||||
|
$(GO) test ./...
|
||||||
|
|
||||||
test-unit: imposm3
|
test-unit: imposm3
|
||||||
$(GO) test ./... -i
|
$(GO) test ./... -i
|
||||||
$(GO) test ./...
|
$(GO) test `$(GO) list ./... | grep -v 'imposm3/test'`
|
||||||
|
|
||||||
test-system: imposm3
|
test-system: imposm3
|
||||||
(cd test && make test)
|
(cd test && make test)
|
||||||
|
|
54
README.md
54
README.md
|
@ -10,8 +10,8 @@ It is designed to create databases that are optimized for rendering (i.e. genera
|
||||||
Imposm 3 is written in Go and it is a complete rewrite of the previous Python implementation.
|
Imposm 3 is written in Go and it is a complete rewrite of the previous Python implementation.
|
||||||
Configurations/mappings and cache files are not compatible with Imposm 2, but they share a similar architecture.
|
Configurations/mappings and cache files are not compatible with Imposm 2, but they share a similar architecture.
|
||||||
|
|
||||||
The development of Imposm 3 was sponsored by [Omniscale](http://omniscale.com/) and development will continue as resources permit.
|
The development of Imposm 3 was sponsored by [Omniscale](http://omniscale.com/). There are [commercial licenses available for Imposm](http://omniscale.com/opensource/soss) to support the long-term development of Imposm.
|
||||||
Please get in touch if you need commercial support or if you need specific features.
|
There is also commercial support available from Omniscale.
|
||||||
|
|
||||||
|
|
||||||
Features
|
Features
|
||||||
|
@ -45,10 +45,13 @@ Features
|
||||||
Automatically creates tables with lower spatial resolutions, perfect for rendering large road networks in low resolutions.
|
Automatically creates tables with lower spatial resolutions, perfect for rendering large road networks in low resolutions.
|
||||||
|
|
||||||
- Limit to polygons:
|
- Limit to polygons:
|
||||||
Limit imported geometries to polygons from Shapefiles or GeoJSON, for city/state/country imports.
|
Limit imported geometries to polygons from GeoJSON, for city/state/country imports.
|
||||||
|
|
||||||
- Easy deployment:
|
- Easy deployment:
|
||||||
Single binary with only runtime dependencies to common libs (GEOS, SQLite and LevelDB)
|
Single binary with only runtime dependencies to common libs (GEOS, ProtoBuf and LevelDB)
|
||||||
|
|
||||||
|
- Route relations:
|
||||||
|
Import all relation types including routes.
|
||||||
|
|
||||||
- Support for table namespace (PostgreSQL schema)
|
- Support for table namespace (PostgreSQL schema)
|
||||||
|
|
||||||
|
@ -80,15 +83,19 @@ Import of Europe 11GB PBF with generalized tables:
|
||||||
Current status
|
Current status
|
||||||
--------------
|
--------------
|
||||||
|
|
||||||
Imposm 3 is used in production but there is no official release yet.
|
Imposm 3 is used in production but there is no official 3.0 release yet.
|
||||||
|
|
||||||
### Missing ###
|
### Planned features ###
|
||||||
|
|
||||||
Compared to Imposm 2:
|
There are a few features we like to see in Imposm 3:
|
||||||
|
|
||||||
|
* Automatic download and import of differential files
|
||||||
* Support for other projections than EPSG:3857 or EPSG:4326
|
* Support for other projections than EPSG:3857 or EPSG:4326
|
||||||
* Import of XML files (unlikely to be implemented in the future, use [osmosis](http://wiki.openstreetmap.org/wiki/Osmosis) to convert XML to PBF first)
|
* Improved integration with tile servers (expiration of updated tiles)
|
||||||
* Custom field/filter functions
|
* Custom field/filter functions
|
||||||
|
* Official releases with binaries for more platforms
|
||||||
|
|
||||||
|
There is no roadmap however, as the implementation of these features largely depends on external funding. There are [commercial licenses available for Imposm](http://omniscale.com/opensource/soss) if you like to help with this development.
|
||||||
|
|
||||||
Installation
|
Installation
|
||||||
------------
|
------------
|
||||||
|
@ -166,11 +173,8 @@ Imposm contains a fixed set of the dependencies that are known to work. You need
|
||||||
|
|
||||||
git clone https://github.com/omniscale/imposm3 src/github.com/omniscale/imposm3
|
git clone https://github.com/omniscale/imposm3 src/github.com/omniscale/imposm3
|
||||||
cd src/github.com/omniscale/imposm3
|
cd src/github.com/omniscale/imposm3
|
||||||
godep go install ./...
|
godep go install ./
|
||||||
|
|
||||||
### FreeBSD
|
|
||||||
|
|
||||||
On FreeBSD you can use the ports system: Simply fetch https://github.com/thomersch/imposm3-freebsd and run `make install`.
|
|
||||||
|
|
||||||
Usage
|
Usage
|
||||||
-----
|
-----
|
||||||
|
@ -245,29 +249,23 @@ The GEOS package is released as LGPL3 and is linked dynamically. See LICENSE.bin
|
||||||
|
|
||||||
#### Unit tests ####
|
#### Unit tests ####
|
||||||
|
|
||||||
go test imposm3/...
|
To run all unit tests:
|
||||||
|
|
||||||
|
make test-unit
|
||||||
|
|
||||||
|
Or:
|
||||||
|
|
||||||
|
godep go test ./...
|
||||||
|
|
||||||
|
|
||||||
#### System tests ####
|
#### System tests ####
|
||||||
|
|
||||||
There are system test that import and update OSM data and verify the database content.
|
There are system test that import and update OSM data and verify the database content.
|
||||||
|
You need `osmosis` to create the test PBF files.
|
||||||
##### Dependencies #####
|
There is a Makefile that creates all test files if necessary and then runs the test itself.
|
||||||
|
|
||||||
These tests are written in Python and requires `nose`, `shapely` and `psycopg2`.
|
|
||||||
|
|
||||||
On a recent Ubuntu can install the following packages for that: `python-nose python-shapely python-psycopg2`
|
|
||||||
Or you can [install a Python virtualenv](https://virtualenv.pypa.io/en/latest/installation.html):
|
|
||||||
|
|
||||||
virtualenv imposm3test
|
|
||||||
source imposm3test/bin/activate
|
|
||||||
pip install nose shapely psycopg2
|
|
||||||
|
|
||||||
You also need `osmosis` to create test PBF files.
|
|
||||||
There is a Makefile that (re)builds `imposm3` and creates all test files if necessary and then runs the test itself.
|
|
||||||
|
|
||||||
make test
|
make test
|
||||||
|
|
||||||
Call `make test-system` to skip the unit tests.
|
Call `make test-system` to skip the unit tests.
|
||||||
|
|
||||||
WARNING: It uses your local PostgeSQL database (`import` schema). Change the database with the standard `PGDATABASE`, `PGHOST`, etc. environment variables.
|
WARNING: It uses your local PostgeSQL database (`imposm3testimport`, `imposm3testproduction` and `imposm3testbackup` schema). Change the database with the standard `PGDATABASE`, `PGHOST`, etc. environment variables.
|
||||||
|
|
|
@ -6,7 +6,6 @@ import (
|
||||||
"runtime"
|
"runtime"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"github.com/golang/protobuf/proto"
|
|
||||||
"github.com/omniscale/imposm3/element"
|
"github.com/omniscale/imposm3/element"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
@ -73,103 +72,3 @@ func BenchmarkUnmarshalDeltaCoords(b *testing.B) {
|
||||||
runtime.GC()
|
runtime.GC()
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func BenchmarkMarshalDeltaCoordsProto(b *testing.B) {
|
|
||||||
var buf []byte
|
|
||||||
var err error
|
|
||||||
|
|
||||||
for n := 0; n < b.N; n++ {
|
|
||||||
deltaCoords := packNodes(nodes)
|
|
||||||
buf, err = proto.Marshal(deltaCoords)
|
|
||||||
if err != nil {
|
|
||||||
panic(err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
deltaCoords := &DeltaCoords{}
|
|
||||||
err = proto.Unmarshal(buf, deltaCoords)
|
|
||||||
if err != nil {
|
|
||||||
panic(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
nodes2 := unpackNodes(deltaCoords, nodes)
|
|
||||||
|
|
||||||
compareNodes(b, nodes, nodes2)
|
|
||||||
runtime.GC()
|
|
||||||
|
|
||||||
}
|
|
||||||
|
|
||||||
func BenchmarkUnmarshalDeltaCoordsProto(b *testing.B) {
|
|
||||||
var buf []byte
|
|
||||||
var err error
|
|
||||||
|
|
||||||
deltaCoords := packNodes(nodes)
|
|
||||||
buf, err = proto.Marshal(deltaCoords)
|
|
||||||
if err != nil {
|
|
||||||
panic(err)
|
|
||||||
}
|
|
||||||
var nodes2 []element.Node
|
|
||||||
for n := 0; n < b.N; n++ {
|
|
||||||
deltaCoords := &DeltaCoords{}
|
|
||||||
err = proto.Unmarshal(buf, deltaCoords)
|
|
||||||
if err != nil {
|
|
||||||
panic(err)
|
|
||||||
}
|
|
||||||
nodes2 = unpackNodes(deltaCoords, nodes)
|
|
||||||
}
|
|
||||||
compareNodes(b, nodes, nodes2)
|
|
||||||
runtime.GC()
|
|
||||||
|
|
||||||
}
|
|
||||||
|
|
||||||
func packNodes(nodes []element.Node) *DeltaCoords {
|
|
||||||
var lastLon, lastLat int64
|
|
||||||
var lon, lat int64
|
|
||||||
var lastId int64
|
|
||||||
ids := make([]int64, len(nodes))
|
|
||||||
lons := make([]int64, len(nodes))
|
|
||||||
lats := make([]int64, len(nodes))
|
|
||||||
|
|
||||||
i := 0
|
|
||||||
for _, nd := range nodes {
|
|
||||||
lon = int64(CoordToInt(nd.Long))
|
|
||||||
lat = int64(CoordToInt(nd.Lat))
|
|
||||||
ids[i] = nd.Id - lastId
|
|
||||||
lons[i] = lon - lastLon
|
|
||||||
lats[i] = lat - lastLat
|
|
||||||
|
|
||||||
lastId = nd.Id
|
|
||||||
lastLon = lon
|
|
||||||
lastLat = lat
|
|
||||||
i++
|
|
||||||
}
|
|
||||||
return &DeltaCoords{Ids: ids, Lats: lats, Lons: lons}
|
|
||||||
}
|
|
||||||
|
|
||||||
func unpackNodes(deltaCoords *DeltaCoords, nodes []element.Node) []element.Node {
|
|
||||||
if len(deltaCoords.Ids) > cap(nodes) {
|
|
||||||
nodes = make([]element.Node, len(deltaCoords.Ids))
|
|
||||||
} else {
|
|
||||||
nodes = nodes[:len(deltaCoords.Ids)]
|
|
||||||
}
|
|
||||||
|
|
||||||
var lastLon, lastLat int64
|
|
||||||
var lon, lat int64
|
|
||||||
var lastId, id int64
|
|
||||||
|
|
||||||
for i := 0; i < len(deltaCoords.Ids); i++ {
|
|
||||||
id = lastId + deltaCoords.Ids[i]
|
|
||||||
lon = lastLon + deltaCoords.Lons[i]
|
|
||||||
lat = lastLat + deltaCoords.Lats[i]
|
|
||||||
nodes[i] = element.Node{
|
|
||||||
OSMElem: element.OSMElem{Id: int64(id)},
|
|
||||||
Long: IntToCoord(uint32(lon)),
|
|
||||||
Lat: IntToCoord(uint32(lat)),
|
|
||||||
}
|
|
||||||
|
|
||||||
lastId = id
|
|
||||||
lastLon = lon
|
|
||||||
lastLat = lat
|
|
||||||
}
|
|
||||||
return nodes
|
|
||||||
}
|
|
||||||
|
|
|
@ -1,8 +1,9 @@
|
||||||
package binary
|
package binary
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"github.com/omniscale/imposm3/element"
|
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
|
"github.com/omniscale/imposm3/element"
|
||||||
)
|
)
|
||||||
|
|
||||||
func compareRefs(a []int64, b []int64) bool {
|
func compareRefs(a []int64, b []int64) bool {
|
||||||
|
@ -74,8 +75,8 @@ func TestMarshalRelation(t *testing.T) {
|
||||||
rel.Tags = make(element.Tags)
|
rel.Tags = make(element.Tags)
|
||||||
rel.Tags["name"] = "test"
|
rel.Tags["name"] = "test"
|
||||||
rel.Tags["landusage"] = "forest"
|
rel.Tags["landusage"] = "forest"
|
||||||
rel.Members = append(rel.Members, element.Member{123, element.WAY, "outer", nil})
|
rel.Members = append(rel.Members, element.Member{Id: 123, Type: element.WAY, Role: "outer"})
|
||||||
rel.Members = append(rel.Members, element.Member{124, element.WAY, "inner", nil})
|
rel.Members = append(rel.Members, element.Member{Id: 124, Type: element.WAY, Role: "inner"})
|
||||||
|
|
||||||
data, _ := MarshalRelation(rel)
|
data, _ := MarshalRelation(rel)
|
||||||
rel, _ = UnmarshalRelation(data)
|
rel, _ = UnmarshalRelation(data)
|
||||||
|
|
|
@ -15,58 +15,6 @@ func (s byId) Len() int { return len(s) }
|
||||||
func (s byId) Swap(i, j int) { s[i], s[j] = s[j], s[i] }
|
func (s byId) Swap(i, j int) { s[i], s[j] = s[j], s[i] }
|
||||||
func (s byId) Less(i, j int) bool { return s[i].Id < s[j].Id }
|
func (s byId) Less(i, j int) bool { return s[i].Id < s[j].Id }
|
||||||
|
|
||||||
func packNodes(nodes []element.Node) *binary.DeltaCoords {
|
|
||||||
var lastLon, lastLat int64
|
|
||||||
var lon, lat int64
|
|
||||||
var lastId int64
|
|
||||||
ids := make([]int64, len(nodes))
|
|
||||||
lons := make([]int64, len(nodes))
|
|
||||||
lats := make([]int64, len(nodes))
|
|
||||||
|
|
||||||
i := 0
|
|
||||||
for _, nd := range nodes {
|
|
||||||
lon = int64(binary.CoordToInt(nd.Long))
|
|
||||||
lat = int64(binary.CoordToInt(nd.Lat))
|
|
||||||
ids[i] = nd.Id - lastId
|
|
||||||
lons[i] = lon - lastLon
|
|
||||||
lats[i] = lat - lastLat
|
|
||||||
|
|
||||||
lastId = nd.Id
|
|
||||||
lastLon = lon
|
|
||||||
lastLat = lat
|
|
||||||
i++
|
|
||||||
}
|
|
||||||
return &binary.DeltaCoords{Ids: ids, Lats: lats, Lons: lons}
|
|
||||||
}
|
|
||||||
|
|
||||||
func unpackNodes(deltaCoords *binary.DeltaCoords, nodes []element.Node) []element.Node {
|
|
||||||
if len(deltaCoords.Ids) > cap(nodes) {
|
|
||||||
nodes = make([]element.Node, len(deltaCoords.Ids))
|
|
||||||
} else {
|
|
||||||
nodes = nodes[:len(deltaCoords.Ids)]
|
|
||||||
}
|
|
||||||
|
|
||||||
var lastLon, lastLat int64
|
|
||||||
var lon, lat int64
|
|
||||||
var lastId, id int64
|
|
||||||
|
|
||||||
for i := 0; i < len(deltaCoords.Ids); i++ {
|
|
||||||
id = lastId + deltaCoords.Ids[i]
|
|
||||||
lon = lastLon + deltaCoords.Lons[i]
|
|
||||||
lat = lastLat + deltaCoords.Lats[i]
|
|
||||||
nodes[i] = element.Node{
|
|
||||||
OSMElem: element.OSMElem{Id: int64(id)},
|
|
||||||
Long: binary.IntToCoord(uint32(lon)),
|
|
||||||
Lat: binary.IntToCoord(uint32(lat)),
|
|
||||||
}
|
|
||||||
|
|
||||||
lastId = id
|
|
||||||
lastLon = lon
|
|
||||||
lastLat = lat
|
|
||||||
}
|
|
||||||
return nodes
|
|
||||||
}
|
|
||||||
|
|
||||||
type coordsBunch struct {
|
type coordsBunch struct {
|
||||||
sync.Mutex
|
sync.Mutex
|
||||||
id int64
|
id int64
|
||||||
|
@ -80,7 +28,8 @@ func (b *coordsBunch) GetCoord(id int64) (*element.Node, error) {
|
||||||
return b.coords[i].Id >= id
|
return b.coords[i].Id >= id
|
||||||
})
|
})
|
||||||
if idx < len(b.coords) && b.coords[idx].Id == id {
|
if idx < len(b.coords) && b.coords[idx].Id == id {
|
||||||
return &b.coords[idx], nil
|
nd := b.coords[idx] // create copy prevent to race when node gets reprojected
|
||||||
|
return &nd, nil
|
||||||
}
|
}
|
||||||
return nil, NotFound
|
return nil, NotFound
|
||||||
}
|
}
|
||||||
|
@ -322,10 +271,6 @@ func (self *DeltaCoordsCache) PutCoords(nodes []element.Node) error {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
var (
|
|
||||||
freeBuffer = make(chan []byte, 4)
|
|
||||||
)
|
|
||||||
|
|
||||||
func (p *DeltaCoordsCache) putCoordsPacked(bunchId int64, nodes []element.Node) error {
|
func (p *DeltaCoordsCache) putCoordsPacked(bunchId int64, nodes []element.Node) error {
|
||||||
keyBuf := idToKeyBuf(bunchId)
|
keyBuf := idToKeyBuf(bunchId)
|
||||||
|
|
||||||
|
@ -333,12 +278,7 @@ func (p *DeltaCoordsCache) putCoordsPacked(bunchId int64, nodes []element.Node)
|
||||||
return p.db.Delete(p.wo, keyBuf)
|
return p.db.Delete(p.wo, keyBuf)
|
||||||
}
|
}
|
||||||
|
|
||||||
var data []byte
|
data := make([]byte, 512)
|
||||||
select {
|
|
||||||
case data = <-freeBuffer:
|
|
||||||
default:
|
|
||||||
}
|
|
||||||
|
|
||||||
data = binary.MarshalDeltaNodes(nodes, data)
|
data = binary.MarshalDeltaNodes(nodes, data)
|
||||||
|
|
||||||
err := p.db.Put(p.wo, keyBuf, data)
|
err := p.db.Put(p.wo, keyBuf, data)
|
||||||
|
@ -346,11 +286,6 @@ func (p *DeltaCoordsCache) putCoordsPacked(bunchId int64, nodes []element.Node)
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
select {
|
|
||||||
case freeBuffer <- data:
|
|
||||||
default:
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -377,10 +312,6 @@ func (self *DeltaCoordsCache) getBunchId(nodeId int64) int64 {
|
||||||
return nodeId / self.bunchSize
|
return nodeId / self.bunchSize
|
||||||
}
|
}
|
||||||
|
|
||||||
var (
|
|
||||||
freeNodes = make(chan []element.Node, 4)
|
|
||||||
)
|
|
||||||
|
|
||||||
func (self *DeltaCoordsCache) getBunch(bunchId int64) (*coordsBunch, error) {
|
func (self *DeltaCoordsCache) getBunch(bunchId int64) (*coordsBunch, error) {
|
||||||
self.mu.Lock()
|
self.mu.Lock()
|
||||||
bunch, ok := self.table[bunchId]
|
bunch, ok := self.table[bunchId]
|
||||||
|
@ -388,12 +319,7 @@ func (self *DeltaCoordsCache) getBunch(bunchId int64) (*coordsBunch, error) {
|
||||||
needsGet := false
|
needsGet := false
|
||||||
if !ok {
|
if !ok {
|
||||||
elem := self.lruList.PushFront(bunchId)
|
elem := self.lruList.PushFront(bunchId)
|
||||||
select {
|
nodes = make([]element.Node, 0, self.bunchSize)
|
||||||
case nodes = <-freeNodes:
|
|
||||||
nodes = nodes[:0]
|
|
||||||
default:
|
|
||||||
nodes = make([]element.Node, 0, self.bunchSize)
|
|
||||||
}
|
|
||||||
bunch = &coordsBunch{id: bunchId, coords: nodes, elem: elem}
|
bunch = &coordsBunch{id: bunchId, coords: nodes, elem: elem}
|
||||||
needsGet = true
|
needsGet = true
|
||||||
self.table[bunchId] = bunch
|
self.table[bunchId] = bunch
|
||||||
|
@ -429,10 +355,6 @@ func (self *DeltaCoordsCache) CheckCapacity() error {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
select {
|
|
||||||
case freeNodes <- bunch.coords:
|
|
||||||
default:
|
|
||||||
}
|
|
||||||
delete(self.table, bunchId)
|
delete(self.table, bunchId)
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
|
|
|
@ -1,12 +1,13 @@
|
||||||
package cache
|
package cache
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"github.com/omniscale/imposm3/element"
|
|
||||||
"io/ioutil"
|
"io/ioutil"
|
||||||
"math/rand"
|
"math/rand"
|
||||||
"os"
|
"os"
|
||||||
"sort"
|
"sort"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
|
"github.com/omniscale/imposm3/element"
|
||||||
)
|
)
|
||||||
|
|
||||||
func mknode(id int64) element.Node {
|
func mknode(id int64) element.Node {
|
||||||
|
@ -185,3 +186,64 @@ func TestSingleUpdate(t *testing.T) {
|
||||||
insertAndCheck(t, cache, 4, 4, 4)
|
insertAndCheck(t, cache, 4, 4, 4)
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func BenchmarkWriteDeltaCoords(b *testing.B) {
|
||||||
|
b.StopTimer()
|
||||||
|
cache_dir, _ := ioutil.TempDir("", "imposm3_test")
|
||||||
|
defer os.RemoveAll(cache_dir)
|
||||||
|
|
||||||
|
cache, err := newDeltaCoordsCache(cache_dir)
|
||||||
|
if err != nil {
|
||||||
|
b.Fatal()
|
||||||
|
}
|
||||||
|
defer cache.Close()
|
||||||
|
|
||||||
|
nodes := make([]element.Node, 10000)
|
||||||
|
for i := range nodes {
|
||||||
|
nodes[i].Id = rand.Int63n(50000)
|
||||||
|
nodes[i].Long = rand.Float64() - 0.5*360
|
||||||
|
nodes[i].Lat = rand.Float64() - 0.5*180
|
||||||
|
}
|
||||||
|
|
||||||
|
b.StartTimer()
|
||||||
|
for i := 0; i < b.N; i++ {
|
||||||
|
for _, n := range nodes {
|
||||||
|
if err := cache.PutCoords([]element.Node{n}); err != nil {
|
||||||
|
b.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func BenchmarkReadDeltaCoords(b *testing.B) {
|
||||||
|
b.StopTimer()
|
||||||
|
cache_dir, _ := ioutil.TempDir("", "imposm3_test")
|
||||||
|
defer os.RemoveAll(cache_dir)
|
||||||
|
|
||||||
|
cache, err := newDeltaCoordsCache(cache_dir)
|
||||||
|
if err != nil {
|
||||||
|
b.Fatal()
|
||||||
|
}
|
||||||
|
defer cache.Close()
|
||||||
|
|
||||||
|
nodes := make([]element.Node, 10000)
|
||||||
|
for i := range nodes {
|
||||||
|
nodes[i].Id = rand.Int63n(50000)
|
||||||
|
nodes[i].Long = rand.Float64() - 0.5*360
|
||||||
|
nodes[i].Lat = rand.Float64() - 0.5*180
|
||||||
|
}
|
||||||
|
for _, n := range nodes {
|
||||||
|
if err := cache.PutCoords([]element.Node{n}); err != nil {
|
||||||
|
b.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
b.StartTimer()
|
||||||
|
for i := 0; i < b.N; i++ {
|
||||||
|
for n := 0; n < 10000; n++ {
|
||||||
|
if _, err := cache.GetCoord(int64(n)); err != nil && err != NotFound {
|
||||||
|
b.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
|
@ -1,7 +1,6 @@
|
||||||
package cache
|
package cache
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"github.com/jmhodges/levigo"
|
|
||||||
"log"
|
"log"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
@ -9,6 +8,8 @@ import (
|
||||||
"sort"
|
"sort"
|
||||||
"sync"
|
"sync"
|
||||||
|
|
||||||
|
"github.com/jmhodges/levigo"
|
||||||
|
|
||||||
"github.com/omniscale/imposm3/cache/binary"
|
"github.com/omniscale/imposm3/cache/binary"
|
||||||
"github.com/omniscale/imposm3/element"
|
"github.com/omniscale/imposm3/element"
|
||||||
)
|
)
|
||||||
|
@ -20,10 +21,11 @@ func (a byInt64) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
|
||||||
func (a byInt64) Less(i, j int) bool { return a[i] < a[j] }
|
func (a byInt64) Less(i, j int) bool { return a[i] < a[j] }
|
||||||
|
|
||||||
type DiffCache struct {
|
type DiffCache struct {
|
||||||
Dir string
|
Dir string
|
||||||
Coords *CoordsRefIndex
|
Coords *CoordsRefIndex // Stores which ways a coord references
|
||||||
Ways *WaysRefIndex
|
CoordsRel *CoordsRelRefIndex // Stores which relations a coord references
|
||||||
opened bool
|
Ways *WaysRefIndex // Stores which relations a way references
|
||||||
|
opened bool
|
||||||
}
|
}
|
||||||
|
|
||||||
func NewDiffCache(dir string) *DiffCache {
|
func NewDiffCache(dir string) *DiffCache {
|
||||||
|
@ -36,6 +38,10 @@ func (c *DiffCache) Close() {
|
||||||
c.Coords.Close()
|
c.Coords.Close()
|
||||||
c.Coords = nil
|
c.Coords = nil
|
||||||
}
|
}
|
||||||
|
if c.CoordsRel != nil {
|
||||||
|
c.CoordsRel.Close()
|
||||||
|
c.CoordsRel = nil
|
||||||
|
}
|
||||||
if c.Ways != nil {
|
if c.Ways != nil {
|
||||||
c.Ways.Close()
|
c.Ways.Close()
|
||||||
c.Ways = nil
|
c.Ways = nil
|
||||||
|
@ -46,6 +52,9 @@ func (c *DiffCache) Flush() {
|
||||||
if c.Coords != nil {
|
if c.Coords != nil {
|
||||||
c.Coords.Flush()
|
c.Coords.Flush()
|
||||||
}
|
}
|
||||||
|
if c.CoordsRel != nil {
|
||||||
|
c.CoordsRel.Flush()
|
||||||
|
}
|
||||||
if c.Ways != nil {
|
if c.Ways != nil {
|
||||||
c.Ways.Flush()
|
c.Ways.Flush()
|
||||||
}
|
}
|
||||||
|
@ -58,6 +67,11 @@ func (c *DiffCache) Open() error {
|
||||||
c.Close()
|
c.Close()
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
c.CoordsRel, err = newCoordsRelRefIndex(filepath.Join(c.Dir, "coords_rel_index"))
|
||||||
|
if err != nil {
|
||||||
|
c.Close()
|
||||||
|
return err
|
||||||
|
}
|
||||||
c.Ways, err = newWaysRefIndex(filepath.Join(c.Dir, "ways_index"))
|
c.Ways, err = newWaysRefIndex(filepath.Join(c.Dir, "ways_index"))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
c.Close()
|
c.Close()
|
||||||
|
@ -74,6 +88,9 @@ func (c *DiffCache) Exists() bool {
|
||||||
if _, err := os.Stat(filepath.Join(c.Dir, "coords_index")); !os.IsNotExist(err) {
|
if _, err := os.Stat(filepath.Join(c.Dir, "coords_index")); !os.IsNotExist(err) {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(c.Dir, "coords_rel_index")); !os.IsNotExist(err) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
if _, err := os.Stat(filepath.Join(c.Dir, "ways_index")); !os.IsNotExist(err) {
|
if _, err := os.Stat(filepath.Join(c.Dir, "ways_index")); !os.IsNotExist(err) {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
@ -87,6 +104,9 @@ func (c *DiffCache) Remove() error {
|
||||||
if err := os.RemoveAll(filepath.Join(c.Dir, "coords_index")); err != nil {
|
if err := os.RemoveAll(filepath.Join(c.Dir, "coords_index")); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
if err := os.RemoveAll(filepath.Join(c.Dir, "coords_rel_index")); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
if err := os.RemoveAll(filepath.Join(c.Dir, "ways_index")); err != nil {
|
if err := os.RemoveAll(filepath.Join(c.Dir, "ways_index")); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
@ -198,6 +218,9 @@ func newRefIndex(path string, opts *cacheOptions) (*bunchRefCache, error) {
|
||||||
type CoordsRefIndex struct {
|
type CoordsRefIndex struct {
|
||||||
bunchRefCache
|
bunchRefCache
|
||||||
}
|
}
|
||||||
|
type CoordsRelRefIndex struct {
|
||||||
|
bunchRefCache
|
||||||
|
}
|
||||||
type WaysRefIndex struct {
|
type WaysRefIndex struct {
|
||||||
bunchRefCache
|
bunchRefCache
|
||||||
}
|
}
|
||||||
|
@ -210,6 +233,14 @@ func newCoordsRefIndex(dir string) (*CoordsRefIndex, error) {
|
||||||
return &CoordsRefIndex{*cache}, nil
|
return &CoordsRefIndex{*cache}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func newCoordsRelRefIndex(dir string) (*CoordsRelRefIndex, error) {
|
||||||
|
cache, err := newRefIndex(dir, &globalCacheOptions.CoordsIndex)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &CoordsRelRefIndex{*cache}, nil
|
||||||
|
}
|
||||||
|
|
||||||
func newWaysRefIndex(dir string) (*WaysRefIndex, error) {
|
func newWaysRefIndex(dir string) (*WaysRefIndex, error) {
|
||||||
cache, err := newRefIndex(dir, &globalCacheOptions.WaysIndex)
|
cache, err := newRefIndex(dir, &globalCacheOptions.WaysIndex)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
@ -365,6 +396,18 @@ func (index *CoordsRefIndex) DeleteFromWay(way *element.Way) {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (index *CoordsRelRefIndex) AddFromMembers(relId int64, members []element.Member) {
|
||||||
|
for _, member := range members {
|
||||||
|
if member.Type == element.NODE {
|
||||||
|
if index.linearImport {
|
||||||
|
index.addc <- idRef{id: member.Id, ref: relId}
|
||||||
|
} else {
|
||||||
|
index.Add(member.Id, relId)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func (index *WaysRefIndex) AddFromMembers(relId int64, members []element.Member) {
|
func (index *WaysRefIndex) AddFromMembers(relId int64, members []element.Member) {
|
||||||
for _, member := range members {
|
for _, member := range members {
|
||||||
if member.Type == element.WAY {
|
if member.Type == element.WAY {
|
||||||
|
|
|
@ -90,9 +90,10 @@ func BenchmarkWriteDiff(b *testing.B) {
|
||||||
for i := 0; i < b.N; i++ {
|
for i := 0; i < b.N; i++ {
|
||||||
for w := 0; w < 5; w++ {
|
for w := 0; w < 5; w++ {
|
||||||
for n := 0; n < 200; n++ {
|
for n := 0; n < 200; n++ {
|
||||||
cache.addc <- idRef{id: int64(n), ref: int64(w)}
|
cache.Add(int64(n), int64(w))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
cache.Flush()
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
47
cmd/main.go
47
cmd/main.go
|
@ -6,12 +6,9 @@ import (
|
||||||
"os"
|
"os"
|
||||||
"runtime"
|
"runtime"
|
||||||
|
|
||||||
"github.com/omniscale/imposm3/cache"
|
|
||||||
|
|
||||||
"github.com/omniscale/imposm3/cache/query"
|
"github.com/omniscale/imposm3/cache/query"
|
||||||
"github.com/omniscale/imposm3/config"
|
"github.com/omniscale/imposm3/config"
|
||||||
"github.com/omniscale/imposm3/diff"
|
"github.com/omniscale/imposm3/diff"
|
||||||
"github.com/omniscale/imposm3/geom/limit"
|
|
||||||
"github.com/omniscale/imposm3/import_"
|
"github.com/omniscale/imposm3/import_"
|
||||||
"github.com/omniscale/imposm3/logging"
|
"github.com/omniscale/imposm3/logging"
|
||||||
"github.com/omniscale/imposm3/stats"
|
"github.com/omniscale/imposm3/stats"
|
||||||
|
@ -53,49 +50,7 @@ func Main(usage func()) {
|
||||||
if config.BaseOptions.Httpprofile != "" {
|
if config.BaseOptions.Httpprofile != "" {
|
||||||
stats.StartHttpPProf(config.BaseOptions.Httpprofile)
|
stats.StartHttpPProf(config.BaseOptions.Httpprofile)
|
||||||
}
|
}
|
||||||
|
diff.Diff()
|
||||||
if config.BaseOptions.Quiet {
|
|
||||||
logging.SetQuiet(true)
|
|
||||||
}
|
|
||||||
|
|
||||||
var geometryLimiter *limit.Limiter
|
|
||||||
if config.BaseOptions.LimitTo != "" {
|
|
||||||
var err error
|
|
||||||
step := log.StartStep("Reading limitto geometries")
|
|
||||||
geometryLimiter, err = limit.NewFromGeoJSON(
|
|
||||||
config.BaseOptions.LimitTo,
|
|
||||||
config.BaseOptions.LimitToCacheBuffer,
|
|
||||||
config.BaseOptions.Srid,
|
|
||||||
)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
log.StopStep(step)
|
|
||||||
}
|
|
||||||
osmCache := cache.NewOSMCache(config.BaseOptions.CacheDir)
|
|
||||||
err := osmCache.Open()
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal("osm cache: ", err)
|
|
||||||
}
|
|
||||||
defer osmCache.Close()
|
|
||||||
|
|
||||||
diffCache := cache.NewDiffCache(config.BaseOptions.CacheDir)
|
|
||||||
err = diffCache.Open()
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal("diff cache: ", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, oscFile := range config.DiffFlags.Args() {
|
|
||||||
err := diff.Update(oscFile, geometryLimiter, nil, osmCache, diffCache, false)
|
|
||||||
if err != nil {
|
|
||||||
osmCache.Close()
|
|
||||||
diffCache.Close()
|
|
||||||
log.Fatalf("unable to process %s: %v", oscFile, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// explicitly Close since os.Exit prevents defers
|
|
||||||
osmCache.Close()
|
|
||||||
diffCache.Close()
|
|
||||||
|
|
||||||
case "query-cache":
|
case "query-cache":
|
||||||
query.Query(os.Args[2:])
|
query.Query(os.Args[2:])
|
||||||
|
|
|
@ -36,6 +36,7 @@ type Inserter interface {
|
||||||
InsertPoint(element.OSMElem, geom.Geometry, []mapping.Match) error
|
InsertPoint(element.OSMElem, geom.Geometry, []mapping.Match) error
|
||||||
InsertLineString(element.OSMElem, geom.Geometry, []mapping.Match) error
|
InsertLineString(element.OSMElem, geom.Geometry, []mapping.Match) error
|
||||||
InsertPolygon(element.OSMElem, geom.Geometry, []mapping.Match) error
|
InsertPolygon(element.OSMElem, geom.Geometry, []mapping.Match) error
|
||||||
|
InsertRelationMember(element.Relation, element.Member, geom.Geometry, []mapping.Match) error
|
||||||
}
|
}
|
||||||
|
|
||||||
type Deployer interface {
|
type Deployer interface {
|
||||||
|
@ -103,6 +104,9 @@ func (n *nullDb) Abort() error
|
||||||
func (n *nullDb) InsertPoint(element.OSMElem, geom.Geometry, []mapping.Match) error { return nil }
|
func (n *nullDb) InsertPoint(element.OSMElem, geom.Geometry, []mapping.Match) error { return nil }
|
||||||
func (n *nullDb) InsertLineString(element.OSMElem, geom.Geometry, []mapping.Match) error { return nil }
|
func (n *nullDb) InsertLineString(element.OSMElem, geom.Geometry, []mapping.Match) error { return nil }
|
||||||
func (n *nullDb) InsertPolygon(element.OSMElem, geom.Geometry, []mapping.Match) error { return nil }
|
func (n *nullDb) InsertPolygon(element.OSMElem, geom.Geometry, []mapping.Match) error { return nil }
|
||||||
|
func (n *nullDb) InsertRelationMember(element.Relation, element.Member, geom.Geometry, []mapping.Match) error {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
func newNullDb(conf Config, m *mapping.Mapping) (DB, error) {
|
func newNullDb(conf Config, m *mapping.Mapping) (DB, error) {
|
||||||
return &nullDb{}, nil
|
return &nullDb{}, nil
|
||||||
|
|
|
@ -58,7 +58,7 @@ func createTable(tx *sql.Tx, spec TableSpec) error {
|
||||||
}
|
}
|
||||||
|
|
||||||
func addGeometryColumn(tx *sql.Tx, tableName string, spec TableSpec) error {
|
func addGeometryColumn(tx *sql.Tx, tableName string, spec TableSpec) error {
|
||||||
colName := "geometry"
|
colName := ""
|
||||||
for _, col := range spec.Columns {
|
for _, col := range spec.Columns {
|
||||||
if col.Type.Name() == "GEOMETRY" {
|
if col.Type.Name() == "GEOMETRY" {
|
||||||
colName = col.Name
|
colName = col.Name
|
||||||
|
@ -66,6 +66,10 @@ func addGeometryColumn(tx *sql.Tx, tableName string, spec TableSpec) error {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if colName == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
geomType := strings.ToUpper(spec.GeometryType)
|
geomType := strings.ToUpper(spec.GeometryType)
|
||||||
if geomType == "POLYGON" {
|
if geomType == "POLYGON" {
|
||||||
geomType = "GEOMETRY" // for multipolygon support
|
geomType = "GEOMETRY" // for multipolygon support
|
||||||
|
@ -203,8 +207,8 @@ func createIndex(pg *PostGIS, tableName string, columns []ColumnSpec) error {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if col.FieldType.Name == "id" {
|
if col.FieldType.Name == "id" {
|
||||||
sql := fmt.Sprintf(`CREATE INDEX "%s_osm_id_idx" ON "%s"."%s" USING BTREE ("%s")`,
|
sql := fmt.Sprintf(`CREATE INDEX "%s_%s_idx" ON "%s"."%s" USING BTREE ("%s")`,
|
||||||
tableName, pg.Config.ImportSchema, tableName, col.Name)
|
tableName, col.Name, pg.Config.ImportSchema, tableName, col.Name)
|
||||||
step := log.StartStep(fmt.Sprintf("Creating OSM id index on %s", tableName))
|
step := log.StartStep(fmt.Sprintf("Creating OSM id index on %s", tableName))
|
||||||
_, err := pg.Db.Exec(sql)
|
_, err := pg.Db.Exec(sql)
|
||||||
log.StopStep(step)
|
log.StopStep(step)
|
||||||
|
@ -480,6 +484,16 @@ func (pg *PostGIS) InsertPolygon(elem element.OSMElem, geom geom.Geometry, match
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (pg *PostGIS) InsertRelationMember(rel element.Relation, m element.Member, geom geom.Geometry, matches []mapping.Match) error {
|
||||||
|
for _, match := range matches {
|
||||||
|
row := match.MemberRow(&rel, &m, &geom)
|
||||||
|
if err := pg.txRouter.Insert(match.Table.Name, row); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
func (pg *PostGIS) Delete(id int64, matches interface{}) error {
|
func (pg *PostGIS) Delete(id int64, matches interface{}) error {
|
||||||
if matches, ok := matches.([]mapping.Match); ok {
|
if matches, ok := matches.([]mapping.Match); ok {
|
||||||
for _, match := range matches {
|
for _, match := range matches {
|
||||||
|
@ -498,10 +512,10 @@ func (pg *PostGIS) DeleteElem(elem element.OSMElem) error {
|
||||||
// handle deletes of geometries that did not match in ProbeXxx.
|
// handle deletes of geometries that did not match in ProbeXxx.
|
||||||
// we have to handle multipolygon relations that took the tags of the
|
// we have to handle multipolygon relations that took the tags of the
|
||||||
// main-member. those tags are not avail. during delete. just try to
|
// main-member. those tags are not avail. during delete. just try to
|
||||||
// delete from each polygon table.
|
// delete from each polygon/relation table.
|
||||||
if v, ok := elem.Tags["type"]; ok && (v == "multipolygon" || v == "boundary") {
|
if _, ok := elem.Tags["type"]; ok {
|
||||||
for _, tableSpec := range pg.Tables {
|
for _, tableSpec := range pg.Tables {
|
||||||
if tableSpec.GeometryType != "polygon" {
|
if tableSpec.GeometryType != "polygon" && tableSpec.GeometryType != "geometry" && tableSpec.GeometryType != "relation" {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
pg.txRouter.Delete(tableSpec.Name, elem.Id)
|
pg.txRouter.Delete(tableSpec.Name, elem.Id)
|
||||||
|
|
|
@ -125,11 +125,19 @@ func (spec *TableSpec) DeleteSQL() string {
|
||||||
}
|
}
|
||||||
|
|
||||||
func NewTableSpec(pg *PostGIS, t *mapping.Table) *TableSpec {
|
func NewTableSpec(pg *PostGIS, t *mapping.Table) *TableSpec {
|
||||||
|
var geomType string
|
||||||
|
switch t.Type {
|
||||||
|
case mapping.RelationMemberTable:
|
||||||
|
geomType = "geometry"
|
||||||
|
default:
|
||||||
|
geomType = string(t.Type)
|
||||||
|
}
|
||||||
|
|
||||||
spec := TableSpec{
|
spec := TableSpec{
|
||||||
Name: t.Name,
|
Name: t.Name,
|
||||||
FullName: pg.Prefix + t.Name,
|
FullName: pg.Prefix + t.Name,
|
||||||
Schema: pg.Config.ImportSchema,
|
Schema: pg.Config.ImportSchema,
|
||||||
GeometryType: string(t.Type),
|
GeometryType: geomType,
|
||||||
Srid: pg.Config.Srid,
|
Srid: pg.Config.Srid,
|
||||||
}
|
}
|
||||||
for _, field := range t.Fields {
|
for _, field := range t.Fields {
|
||||||
|
|
|
@ -83,18 +83,12 @@ func (d *Deleter) deleteRelation(id int64, deleteRefs bool, deleteMembers bool)
|
||||||
if elem.Tags == nil {
|
if elem.Tags == nil {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
if matches := d.tmPolygons.MatchRelation(elem); len(matches) > 0 {
|
// delete from all tables to handle relations with tags from members
|
||||||
if err := d.delDb.Delete(d.RelId(elem.Id), matches); err != nil {
|
// and relation_members
|
||||||
return err
|
e := element.OSMElem(elem.OSMElem)
|
||||||
}
|
e.Id = -e.Id
|
||||||
} else {
|
if err := d.delDb.DeleteElem(e); err != nil {
|
||||||
// handle relations with tags from members by deleting
|
return err
|
||||||
// from all tables
|
|
||||||
e := element.OSMElem(elem.OSMElem)
|
|
||||||
e.Id = -e.Id
|
|
||||||
if err := d.delDb.DeleteElem(e); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if deleteRefs {
|
if deleteRefs {
|
||||||
|
@ -116,8 +110,10 @@ func (d *Deleter) deleteRelation(id int64, deleteRefs bool, deleteMembers bool)
|
||||||
if _, ok := d.deletedWays[member.Id]; ok {
|
if _, ok := d.deletedWays[member.Id]; ok {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if err := d.deleteRelation(member.Id, false, false); err != nil {
|
for _, r := range d.diffCache.Ways.Get(member.Id) {
|
||||||
return err
|
if err := d.deleteRelation(r, false, false); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if err := d.deleteWay(member.Id, false); err != nil {
|
if err := d.deleteWay(member.Id, false); err != nil {
|
||||||
return err
|
return err
|
||||||
|
@ -262,6 +258,15 @@ func (d *Deleter) Delete(delElem parser.DiffElem) error {
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
dependers = d.diffCache.CoordsRel.Get(delElem.Node.Id)
|
||||||
|
for _, rel := range dependers {
|
||||||
|
if _, ok := d.deletedRelations[rel]; ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err := d.deleteRelation(rel, false, false); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if !delElem.Add {
|
if !delElem.Add {
|
||||||
if err := d.diffCache.Coords.Delete(delElem.Node.Id); err != nil {
|
if err := d.diffCache.Coords.Delete(delElem.Node.Id); err != nil {
|
||||||
|
|
|
@ -24,6 +24,51 @@ import (
|
||||||
|
|
||||||
var log = logging.NewLogger("diff")
|
var log = logging.NewLogger("diff")
|
||||||
|
|
||||||
|
func Diff() {
|
||||||
|
if config.BaseOptions.Quiet {
|
||||||
|
logging.SetQuiet(true)
|
||||||
|
}
|
||||||
|
|
||||||
|
var geometryLimiter *limit.Limiter
|
||||||
|
if config.BaseOptions.LimitTo != "" {
|
||||||
|
var err error
|
||||||
|
step := log.StartStep("Reading limitto geometries")
|
||||||
|
geometryLimiter, err = limit.NewFromGeoJSON(
|
||||||
|
config.BaseOptions.LimitTo,
|
||||||
|
config.BaseOptions.LimitToCacheBuffer,
|
||||||
|
config.BaseOptions.Srid,
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
log.StopStep(step)
|
||||||
|
}
|
||||||
|
osmCache := cache.NewOSMCache(config.BaseOptions.CacheDir)
|
||||||
|
err := osmCache.Open()
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal("osm cache: ", err)
|
||||||
|
}
|
||||||
|
defer osmCache.Close()
|
||||||
|
|
||||||
|
diffCache := cache.NewDiffCache(config.BaseOptions.CacheDir)
|
||||||
|
err = diffCache.Open()
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal("diff cache: ", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, oscFile := range config.DiffFlags.Args() {
|
||||||
|
err := Update(oscFile, geometryLimiter, nil, osmCache, diffCache, false)
|
||||||
|
if err != nil {
|
||||||
|
osmCache.Close()
|
||||||
|
diffCache.Close()
|
||||||
|
log.Fatalf("unable to process %s: %v", oscFile, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// explicitly Close since os.Exit prevents defers
|
||||||
|
osmCache.Close()
|
||||||
|
diffCache.Close()
|
||||||
|
}
|
||||||
|
|
||||||
func Update(oscFile string, geometryLimiter *limit.Limiter, expireor expire.Expireor, osmCache *cache.OSMCache, diffCache *cache.DiffCache, force bool) error {
|
func Update(oscFile string, geometryLimiter *limit.Limiter, expireor expire.Expireor, osmCache *cache.OSMCache, diffCache *cache.DiffCache, force bool) error {
|
||||||
state, err := diffstate.ParseFromOsc(oscFile)
|
state, err := diffstate.ParseFromOsc(oscFile)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
@ -104,6 +149,8 @@ func Update(oscFile string, geometryLimiter *limit.Limiter, expireor expire.Expi
|
||||||
relations,
|
relations,
|
||||||
db, progress,
|
db, progress,
|
||||||
tagmapping.PolygonMatcher(),
|
tagmapping.PolygonMatcher(),
|
||||||
|
tagmapping.RelationMatcher(),
|
||||||
|
tagmapping.RelationMemberMatcher(),
|
||||||
config.BaseOptions.Srid)
|
config.BaseOptions.Srid)
|
||||||
relWriter.SetLimiter(geometryLimiter)
|
relWriter.SetLimiter(geometryLimiter)
|
||||||
relWriter.SetExpireor(expireor)
|
relWriter.SetExpireor(expireor)
|
||||||
|
@ -128,9 +175,9 @@ func Update(oscFile string, geometryLimiter *limit.Limiter, expireor expire.Expi
|
||||||
nodeWriter.SetExpireor(expireor)
|
nodeWriter.SetExpireor(expireor)
|
||||||
nodeWriter.Start()
|
nodeWriter.Start()
|
||||||
|
|
||||||
nodeIds := make(map[int64]bool)
|
nodeIds := make(map[int64]struct{})
|
||||||
wayIds := make(map[int64]bool)
|
wayIds := make(map[int64]struct{})
|
||||||
relIds := make(map[int64]bool)
|
relIds := make(map[int64]struct{})
|
||||||
|
|
||||||
step := log.StartStep("Parsing changes, updating cache and removing elements")
|
step := log.StartStep("Parsing changes, updating cache and removing elements")
|
||||||
|
|
||||||
|
@ -204,7 +251,7 @@ func Update(oscFile string, geometryLimiter *limit.Limiter, expireor expire.Expi
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return diffError(err, "put relation %v", elem.Rel)
|
return diffError(err, "put relation %v", elem.Rel)
|
||||||
}
|
}
|
||||||
relIds[elem.Rel.Id] = true
|
relIds[elem.Rel.Id] = struct{}{}
|
||||||
}
|
}
|
||||||
} else if elem.Way != nil {
|
} else if elem.Way != nil {
|
||||||
// check if first coord is cached to avoid caching
|
// check if first coord is cached to avoid caching
|
||||||
|
@ -218,7 +265,7 @@ func Update(oscFile string, geometryLimiter *limit.Limiter, expireor expire.Expi
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return diffError(err, "put way %v", elem.Way)
|
return diffError(err, "put way %v", elem.Way)
|
||||||
}
|
}
|
||||||
wayIds[elem.Way.Id] = true
|
wayIds[elem.Way.Id] = struct{}{}
|
||||||
}
|
}
|
||||||
} else if elem.Node != nil {
|
} else if elem.Node != nil {
|
||||||
addNode := true
|
addNode := true
|
||||||
|
@ -236,7 +283,7 @@ func Update(oscFile string, geometryLimiter *limit.Limiter, expireor expire.Expi
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return diffError(err, "put coord %v", elem.Node)
|
return diffError(err, "put coord %v", elem.Node)
|
||||||
}
|
}
|
||||||
nodeIds[elem.Node.Id] = true
|
nodeIds[elem.Node.Id] = struct{}{}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
@ -254,7 +301,7 @@ func Update(oscFile string, geometryLimiter *limit.Limiter, expireor expire.Expi
|
||||||
|
|
||||||
// mark member ways from deleted relations for re-insert
|
// mark member ways from deleted relations for re-insert
|
||||||
for id, _ := range deleter.DeletedMemberWays() {
|
for id, _ := range deleter.DeletedMemberWays() {
|
||||||
wayIds[id] = true
|
wayIds[id] = struct{}{}
|
||||||
}
|
}
|
||||||
|
|
||||||
progress.Stop()
|
progress.Stop()
|
||||||
|
@ -267,16 +314,22 @@ func Update(oscFile string, geometryLimiter *limit.Limiter, expireor expire.Expi
|
||||||
for nodeId, _ := range nodeIds {
|
for nodeId, _ := range nodeIds {
|
||||||
dependers := diffCache.Coords.Get(nodeId)
|
dependers := diffCache.Coords.Get(nodeId)
|
||||||
for _, way := range dependers {
|
for _, way := range dependers {
|
||||||
wayIds[way] = true
|
wayIds[way] = struct{}{}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// mark depending relations for (re)insert
|
// mark depending relations for (re)insert
|
||||||
|
for nodeId, _ := range nodeIds {
|
||||||
|
dependers := diffCache.CoordsRel.Get(nodeId)
|
||||||
|
for _, rel := range dependers {
|
||||||
|
relIds[rel] = struct{}{}
|
||||||
|
}
|
||||||
|
}
|
||||||
for wayId, _ := range wayIds {
|
for wayId, _ := range wayIds {
|
||||||
dependers := diffCache.Ways.Get(wayId)
|
dependers := diffCache.Ways.Get(wayId)
|
||||||
// mark depending relations for (re)insert
|
// mark depending relations for (re)insert
|
||||||
for _, rel := range dependers {
|
for _, rel := range dependers {
|
||||||
relIds[rel] = true
|
relIds[rel] = struct{}{}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -290,7 +343,11 @@ func Update(oscFile string, geometryLimiter *limit.Limiter, expireor expire.Expi
|
||||||
}
|
}
|
||||||
// insert new relation
|
// insert new relation
|
||||||
progress.AddRelations(1)
|
progress.AddRelations(1)
|
||||||
relations <- rel
|
// filter out unsupported relation types, otherwise they might
|
||||||
|
// get inserted with the tags from an outer way
|
||||||
|
if relTagFilter.Filter(&rel.Tags) {
|
||||||
|
relations <- rel
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
for wayId, _ := range wayIds {
|
for wayId, _ := range wayIds {
|
||||||
|
|
|
@ -58,6 +58,7 @@ Contents
|
||||||
install
|
install
|
||||||
tutorial
|
tutorial
|
||||||
mapping
|
mapping
|
||||||
|
relations
|
||||||
|
|
||||||
|
|
||||||
.. Indices and tables
|
.. Indices and tables
|
||||||
|
|
|
@ -15,7 +15,7 @@ The most important part is the ``tables`` definition. Each table is a YAML objec
|
||||||
``type``
|
``type``
|
||||||
~~~~~~~~
|
~~~~~~~~
|
||||||
|
|
||||||
``type`` can be ``point``, ``linestring``, ``polygon`` or ``geometry``. ``geometry`` requires a special ``mapping``.
|
``type`` can be ``point``, ``linestring``, ``polygon``, ``geometry``, ``relation`` and ``relation_member``. ``geometry`` requires a special ``mapping``. :doc:`Relations are described in more detail here <relations>`.
|
||||||
|
|
||||||
|
|
||||||
``mapping``
|
``mapping``
|
||||||
|
@ -41,7 +41,7 @@ To import all polygons with `tourism=zoo`, `natural=wood` or `natural=land` into
|
||||||
``columns``
|
``columns``
|
||||||
~~~~~~~~~~~
|
~~~~~~~~~~~
|
||||||
|
|
||||||
``columns`` is a list of columns that Imposm should create for this table. Each column is a YAML object with a ``type`` and a ``name`` and optionaly ``key`` and ``args``.
|
``columns`` is a list of columns that Imposm should create for this table. Each column is a YAML object with a ``type`` and a ``name`` and optionaly ``key``, ``args`` and ``from_member``.
|
||||||
|
|
||||||
``name``
|
``name``
|
||||||
^^^^^^^^^
|
^^^^^^^^^
|
||||||
|
@ -67,6 +67,11 @@ See :ref:`column_types` for documentation of all types.
|
||||||
|
|
||||||
Some column types require additional arguments. Refer to the documentation of the type.
|
Some column types require additional arguments. Refer to the documentation of the type.
|
||||||
|
|
||||||
|
``from_member``
|
||||||
|
^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
``from_member`` is only valid for tables of the type ``relation_member``. If this is set to ``true``, then tags will be used from the member instead of the relation.
|
||||||
|
|
||||||
|
|
||||||
Example
|
Example
|
||||||
~~~~~~~
|
~~~~~~~
|
||||||
|
@ -241,6 +246,34 @@ Stores all tags in a HStore column. Requires the PostGIS HStore extension. This
|
||||||
.. "string_suffixreplace": {"string_suffixreplace", "string", nil, MakeSuffixReplace},
|
.. "string_suffixreplace": {"string_suffixreplace", "string", nil, MakeSuffixReplace},
|
||||||
|
|
||||||
|
|
||||||
|
Element types for ``relation_member``
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
The following types are only valid for tables of the type ``relation_member``.
|
||||||
|
|
||||||
|
``member_id``
|
||||||
|
^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
The OSM ID of the relation member.
|
||||||
|
|
||||||
|
``member_type``
|
||||||
|
^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
The type of the relation member. 0 for nodes, 1 for ways and 2 for relations.
|
||||||
|
|
||||||
|
|
||||||
|
``member_role``
|
||||||
|
^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
The role of the relation member as a string, e.g. `outer`, `stop`, etc.
|
||||||
|
|
||||||
|
|
||||||
|
``member_index``
|
||||||
|
^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
The index of the member in the relation, starting from 0. E.g. the first member is 0, second member is 1, etc.
|
||||||
|
This can be used to query bus stops of a route relation in the right order.
|
||||||
|
|
||||||
|
|
||||||
Generalized Tables
|
Generalized Tables
|
||||||
------------------
|
------------------
|
||||||
|
|
|
@ -0,0 +1,180 @@
|
||||||
|
Relations
|
||||||
|
=========
|
||||||
|
|
||||||
|
In `OpenStreetMap, relations <http://wiki.openstreetmap.org/wiki/Relation>`_ define logical or geographic relationships between other nodes, ways and relations.
|
||||||
|
|
||||||
|
The most common relation type is a multipolygon, but all other relations can be imported as well.
|
||||||
|
|
||||||
|
Multipolygons
|
||||||
|
-------------
|
||||||
|
|
||||||
|
`Multipolygon relations <http://wiki.openstreetmap.org/wiki/Relation:multipolygon>`_ are used to represent complex polygon geometries. They are also the only way to represent holes in polygons.
|
||||||
|
|
||||||
|
|
||||||
|
Multipolygon relations are automatically handled by Imposm for all ``polygon`` tables.
|
||||||
|
|
||||||
|
The following mapping::
|
||||||
|
|
||||||
|
tables:
|
||||||
|
buildings:
|
||||||
|
type: polygon
|
||||||
|
mapping:
|
||||||
|
building: [__any__]
|
||||||
|
|
||||||
|
|
||||||
|
Inserts closed ways if they have a ``building`` tag::
|
||||||
|
|
||||||
|
<way id="1001" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
|
<nd ref="1001"/>
|
||||||
|
...
|
||||||
|
<nd ref="1001"/>
|
||||||
|
<tag k="building" v="yes"/>
|
||||||
|
</way>
|
||||||
|
|
||||||
|
It will also insert relations of the type ``multipolygon`` with a ``building`` tag::
|
||||||
|
|
||||||
|
<relation id="17101" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
|
<member type="way" ref="17101" role="outer"/>
|
||||||
|
<member type="way" ref="17102" role="outer"/>
|
||||||
|
<tag k="type" v="multipolygon"/>
|
||||||
|
<tag k="building" v="yes"/>
|
||||||
|
</relation>
|
||||||
|
|
||||||
|
The roles are ignored by Imposm as not all holes are correctly tagged as ``inner``. Imposm uses geometry operations to verify if a member of a multipolygon is a hole, or if it is a separate polygon.
|
||||||
|
|
||||||
|
|
||||||
|
For compatibility, multipolygon relations without tags will use the tags from the (longest) outer way. Imposm will insert the following relation as well::
|
||||||
|
|
||||||
|
<way id="18101" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
|
<nd ref="1001"/>
|
||||||
|
...
|
||||||
|
<nd ref="1001"/>
|
||||||
|
<tag k="building" v="yes"/>
|
||||||
|
</way>
|
||||||
|
<relation id="18901" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
|
<member type="way" ref="18101" role="outer"/>
|
||||||
|
<member type="way" ref="18102" role="outer"/>
|
||||||
|
<tag k="type" v="multipolygon"/>
|
||||||
|
</relation>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Other relations
|
||||||
|
---------------
|
||||||
|
|
||||||
|
OpenStreetMap also uses relations to map more complex features. Some examples:
|
||||||
|
|
||||||
|
- `Administrative areas <http://wiki.openstreetmap.org/wiki/Relation:boundary>`_ with boundaries, capitals and label positions.
|
||||||
|
- `Bus/tram/train routes <http://wiki.openstreetmap.org/wiki/Relation:route>`_ with the route itself, stops and platforms.
|
||||||
|
- `3D buildings <http://wiki.openstreetmap.org/wiki/Simple_3D_buildings>`_ with multiple parts that should not be computed as holes.
|
||||||
|
|
||||||
|
These relations can not be mapped to `simple` linestrings or polygons as they can contain a mix of different geometry types, or would result in invalid geometries (overlapping polygons).
|
||||||
|
|
||||||
|
The Imposm table types ``relation`` and ``relation_member`` allow you to import all relevant data for these relations.
|
||||||
|
|
||||||
|
``relation_member``
|
||||||
|
^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
The ``relation_member`` table type inserts each member of the relation as a separate row. The ``relation_member`` has access to the `role` and `type` value of each member. You can also import tags from the relation `and` from the member node, way or relation.
|
||||||
|
|
||||||
|
Example
|
||||||
|
~~~~~~~
|
||||||
|
|
||||||
|
You can use the following mapping::
|
||||||
|
|
||||||
|
route_members:
|
||||||
|
type: relation_member
|
||||||
|
columns:
|
||||||
|
- name: osm_id
|
||||||
|
type: id
|
||||||
|
- name: member
|
||||||
|
type: member_id
|
||||||
|
- name: index
|
||||||
|
type: member_index
|
||||||
|
- name: role
|
||||||
|
type: member_role
|
||||||
|
- name: type
|
||||||
|
type: member_type
|
||||||
|
- name: geometry
|
||||||
|
type: geometry
|
||||||
|
- name: relname
|
||||||
|
key: name
|
||||||
|
type: string
|
||||||
|
- name: name
|
||||||
|
key: name
|
||||||
|
type: string
|
||||||
|
from_member: true
|
||||||
|
- key: ref
|
||||||
|
name: ref
|
||||||
|
type: string
|
||||||
|
mapping:
|
||||||
|
route: [bus]
|
||||||
|
|
||||||
|
|
||||||
|
to import a bus relation with stops, a platform and the route itself::
|
||||||
|
|
||||||
|
<relation id="100901" version="1" timestamp="2015-06-02T04:13:19Z">
|
||||||
|
<member type="node" ref="100101" role="stop_entry_only"/>
|
||||||
|
<member type="node" ref="100102" role="stop"/>
|
||||||
|
<member type="way" ref="100511" role="platform"/>
|
||||||
|
<member type="node" ref="100103" role="stop_exit_only"/>
|
||||||
|
<member type="way" ref="100501" role=""/>
|
||||||
|
<member type="way" ref="100502" role=""/>
|
||||||
|
<member type="way" ref="100503" role=""/>
|
||||||
|
<tag k="name" v="Bus 301: A => B"/>
|
||||||
|
<tag k="network" v="ABC"/>
|
||||||
|
<tag k="ref" v="301"/>
|
||||||
|
<tag k="route" v="bus"/>
|
||||||
|
<tag k="type" v="route"/>
|
||||||
|
</relation>
|
||||||
|
|
||||||
|
This will result in seven rows with the following columns:
|
||||||
|
|
||||||
|
======== ======================================================================================================================================================
|
||||||
|
Column Description
|
||||||
|
======== ======================================================================================================================================================
|
||||||
|
osm_id The ID of the relation. 100901 for all members.
|
||||||
|
member The ID of the member. 100101, 100102, etc.
|
||||||
|
index The index of the member. From 1 for 100101 to 7 for 100503. This can be used to query the bus stops in the correct order.
|
||||||
|
role The role of the member. ``stop``, ``platform``, etc.
|
||||||
|
type 0 for nodes, 1 for ways and 2 for other relations.
|
||||||
|
geometry The geometry of the member. Point for nodes and linestring for ways.
|
||||||
|
relname The value of the ``name`` tag of the relation. ``Bus 301: A => B`` in this case.
|
||||||
|
name The value of the ``name`` tag of the member element, if it has one. Note that the mapping contains ``from_member: true`` for this column.
|
||||||
|
ref The value of the ``ref`` tag of the relation. ``301`` in this case.
|
||||||
|
======== ======================================================================================================================================================
|
||||||
|
|
||||||
|
|
||||||
|
You can insert the tags of the relation in a separate ``relation`` table to avoid duplication and then use `joins` when querying the data.
|
||||||
|
Both ``osm_id`` and ``member_id`` columns are indexed in PostgreSQL by default to speed up these joins.
|
||||||
|
|
||||||
|
``relation``
|
||||||
|
^^^^^^^^^^^^
|
||||||
|
|
||||||
|
The ``relation`` table type inserts the mapped element regardless of the resulting geometry. For example, this allows you to create a table with the metadata (name, reference, operator, etc.) of all available route relations. The actual geometries need to be `joined` form the members.
|
||||||
|
|
||||||
|
Example
|
||||||
|
~~~~~~~
|
||||||
|
|
||||||
|
The following mapping imports the bus route relation from above::
|
||||||
|
|
||||||
|
routes:
|
||||||
|
type: relation
|
||||||
|
columns:
|
||||||
|
- name: osm_id
|
||||||
|
type: id
|
||||||
|
- key: ref
|
||||||
|
name: ref
|
||||||
|
type: string
|
||||||
|
- name: network
|
||||||
|
key: network
|
||||||
|
type: string
|
||||||
|
mapping:
|
||||||
|
route: [bus]
|
||||||
|
|
||||||
|
|
||||||
|
This will create a single row with the mapped columns.
|
||||||
|
|
||||||
|
.. note:: ``relation`` tables do not support geometry columns. Use the geometries of the members, or use a ``polygon`` table if your relations contain multipolygons.
|
||||||
|
|
||||||
|
|
|
@ -24,7 +24,7 @@ This is step zero, since you have to do it only once. The following commands cre
|
||||||
createdb -E UTF8 -O osm osm
|
createdb -E UTF8 -O osm osm
|
||||||
psql -d osm -c "CREATE EXTENSION postgis;"
|
psql -d osm -c "CREATE EXTENSION postgis;"
|
||||||
psql -d osm -c "CREATE EXTENSION hstore;" # only required for hstore support
|
psql -d osm -c "CREATE EXTENSION hstore;" # only required for hstore support
|
||||||
echo "ALTER USER osm WITH PASSWORD \'osm\';" |psql -d osm
|
echo "ALTER USER osm WITH PASSWORD 'osm';" |psql -d osm
|
||||||
|
|
||||||
You can change the names if you like, but we will use `osm` for user name, password and database name in all following examples.
|
You can change the names if you like, but we will use `osm` for user name, password and database name in all following examples.
|
||||||
|
|
||||||
|
@ -86,6 +86,11 @@ You can combine reading and writing::
|
||||||
imposm3 import -mapping mapping.yml -read hamburg.osm.pbf -write -connection postgis://osm:osm@localhost/osm
|
imposm3 import -mapping mapping.yml -read hamburg.osm.pbf -write -connection postgis://osm:osm@localhost/osm
|
||||||
|
|
||||||
|
|
||||||
|
All tables are prefixed with ``osm_``, e.g. ``roads`` will create the table ``osm_roads``. You can change the prefix by appending ``?prefix=myprefix`` to the connection URL. Use ``NONE`` to disable prefixing::
|
||||||
|
|
||||||
|
imposm3 import -mapping mapping.yml -write -connection postgis://osm:osm@localhost/osm?prefix=NONE
|
||||||
|
|
||||||
|
|
||||||
Limit to
|
Limit to
|
||||||
~~~~~~~~
|
~~~~~~~~
|
||||||
|
|
||||||
|
@ -204,6 +209,8 @@ To update an existing database with three change files::
|
||||||
|
|
||||||
imposm3 diff -config config.json changes-1.osc.gz changes-2.osc.gz changes-3.osc.gz
|
imposm3 diff -config config.json changes-1.osc.gz changes-2.osc.gz changes-3.osc.gz
|
||||||
|
|
||||||
|
Imposm 3 stores the sequence number of the last imported changeset in `${cachedir}/last.state.txt`, if it finds a matching state file (`123.state.txt` for `123.osc.gz`). Imposm refuses to import the same diff files a second time if these state files are present.
|
||||||
|
|
||||||
Remember that you have to make the initial import with the ``-diff`` option. See above.
|
Remember that you have to make the initial import with the ``-diff`` option. See above.
|
||||||
|
|
||||||
.. note:: You should not make changes to the mapping file after the initial import. Changes are not detected and this can result aborted updates or incomplete data.
|
.. note:: You should not make changes to the mapping file after the initial import. Changes are not detected and this can result aborted updates or incomplete data.
|
||||||
|
|
|
@ -72,6 +72,8 @@ type Member struct {
|
||||||
Type MemberType `json:"type"`
|
Type MemberType `json:"type"`
|
||||||
Role string `json:"role"`
|
Role string `json:"role"`
|
||||||
Way *Way `json:"-"`
|
Way *Way `json:"-"`
|
||||||
|
Node *Node `json:"-"`
|
||||||
|
Elem *OSMElem `json:"-"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type Relation struct {
|
type Relation struct {
|
||||||
|
|
|
@ -1,9 +1,10 @@
|
||||||
package geom
|
package geom
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"testing"
|
||||||
|
|
||||||
"github.com/omniscale/imposm3/element"
|
"github.com/omniscale/imposm3/element"
|
||||||
"github.com/omniscale/imposm3/geom/geos"
|
"github.com/omniscale/imposm3/geom/geos"
|
||||||
"testing"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestLineString(t *testing.T) {
|
func TestLineString(t *testing.T) {
|
||||||
|
|
|
@ -13,9 +13,11 @@ extern void initGEOS_debug();
|
||||||
import "C"
|
import "C"
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"github.com/omniscale/imposm3/logging"
|
"errors"
|
||||||
"runtime"
|
"runtime"
|
||||||
"unsafe"
|
"unsafe"
|
||||||
|
|
||||||
|
"github.com/omniscale/imposm3/logging"
|
||||||
)
|
)
|
||||||
|
|
||||||
var log = logging.NewLogger("GEOS")
|
var log = logging.NewLogger("GEOS")
|
||||||
|
@ -244,6 +246,13 @@ func (this *Geos) IsValid(geom *Geom) bool {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (this *Geos) IsSimple(geom *Geom) bool {
|
||||||
|
if C.GEOSisSimple_r(this.v, geom.v) == 1 {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
func (this *Geos) IsEmpty(geom *Geom) bool {
|
func (this *Geos) IsEmpty(geom *Geom) bool {
|
||||||
if C.GEOSisEmpty_r(this.v, geom.v) == 1 {
|
if C.GEOSisEmpty_r(this.v, geom.v) == 1 {
|
||||||
return true
|
return true
|
||||||
|
@ -268,6 +277,19 @@ func (this *Geos) Equals(a, b *Geom) bool {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (g *Geos) MakeValid(geom *Geom) (*Geom, error) {
|
||||||
|
if g.IsValid(geom) {
|
||||||
|
return geom, nil
|
||||||
|
}
|
||||||
|
fixed := g.Buffer(geom, 0)
|
||||||
|
if fixed == nil {
|
||||||
|
return nil, errors.New("Error while fixing geom with buffer(0)")
|
||||||
|
}
|
||||||
|
g.Destroy(geom)
|
||||||
|
|
||||||
|
return fixed, nil
|
||||||
|
}
|
||||||
|
|
||||||
func (this *Geom) Area() float64 {
|
func (this *Geom) Area() float64 {
|
||||||
var area C.double
|
var area C.double
|
||||||
if ret := C.GEOSArea(this.v, &area); ret == 1 {
|
if ret := C.GEOSArea(this.v, &area); ret == 1 {
|
||||||
|
|
|
@ -93,23 +93,23 @@ func buildRings(rel *element.Relation, maxRingGap float64) ([]*ring, error) {
|
||||||
}
|
}
|
||||||
// merge incomplete rings
|
// merge incomplete rings
|
||||||
mergedRings = mergeRings(incompleteRings)
|
mergedRings = mergeRings(incompleteRings)
|
||||||
if len(completeRings)+len(mergedRings) == 0 {
|
|
||||||
err = ErrorNoRing // for defer
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
// create geometries for merged rings
|
// create geometries for merged rings
|
||||||
for _, ring := range mergedRings {
|
for _, ring := range mergedRings {
|
||||||
if !ring.isClosed() && !ring.tryClose(maxRingGap) {
|
if !ring.isClosed() && !ring.tryClose(maxRingGap) {
|
||||||
err = ErrorNoRing // for defer
|
continue
|
||||||
return nil, err
|
|
||||||
}
|
}
|
||||||
ring.geom, err = Polygon(g, ring.nodes)
|
ring.geom, err = Polygon(g, ring.nodes)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
completeRings = append(completeRings, ring)
|
||||||
}
|
}
|
||||||
|
|
||||||
completeRings = append(completeRings, mergedRings...)
|
if len(completeRings) == 0 {
|
||||||
|
err = ErrorNoRing // for defer
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
// sort by area (large to small)
|
// sort by area (large to small)
|
||||||
for _, r := range completeRings {
|
for _, r := range completeRings {
|
||||||
|
@ -196,13 +196,10 @@ func buildRelGeometry(g *geos.Geos, rel *element.Relation, rings []*ring) (*geos
|
||||||
return nil, errors.New("Error while building multi-polygon.")
|
return nil, errors.New("Error while building multi-polygon.")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if !g.IsValid(result) {
|
var err error
|
||||||
buffered := g.Buffer(result, 0)
|
result, err = g.MakeValid(result)
|
||||||
if buffered == nil {
|
if err != nil {
|
||||||
return nil, errors.New("Error while fixing geom with buffer(0)")
|
return nil, err
|
||||||
}
|
|
||||||
g.Destroy(result)
|
|
||||||
result = buffered
|
|
||||||
}
|
}
|
||||||
|
|
||||||
g.DestroyLater(result)
|
g.DestroyLater(result)
|
||||||
|
|
|
@ -31,7 +31,6 @@ func buildRelation(rel *element.Relation, srid int) (Geometry, error) {
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return Geometry{}, err
|
return Geometry{}, err
|
||||||
}
|
}
|
||||||
|
|
||||||
return prep.Build()
|
return prep.Build()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -54,8 +53,8 @@ func TestSimplePolygonWithHole(t *testing.T) {
|
||||||
rel := element.Relation{
|
rel := element.Relation{
|
||||||
OSMElem: element.OSMElem{Id: 1, Tags: element.Tags{}}}
|
OSMElem: element.OSMElem{Id: 1, Tags: element.Tags{}}}
|
||||||
rel.Members = []element.Member{
|
rel.Members = []element.Member{
|
||||||
{1, element.WAY, "outer", &w1},
|
{Id: 1, Type: element.WAY, Role: "outer", Way: &w1},
|
||||||
{2, element.WAY, "inner", &w2},
|
{Id: 2, Type: element.WAY, Role: "inner", Way: &w2},
|
||||||
}
|
}
|
||||||
|
|
||||||
geom, err := buildRelation(&rel, 3857)
|
geom, err := buildRelation(&rel, 3857)
|
||||||
|
@ -97,8 +96,8 @@ func TestMultiPolygonWithHoleAndRelName(t *testing.T) {
|
||||||
rel := element.Relation{
|
rel := element.Relation{
|
||||||
OSMElem: element.OSMElem{Id: 1, Tags: element.Tags{"name": "rel"}}}
|
OSMElem: element.OSMElem{Id: 1, Tags: element.Tags{"name": "rel"}}}
|
||||||
rel.Members = []element.Member{
|
rel.Members = []element.Member{
|
||||||
{1, element.WAY, "outer", &w1},
|
{Id: 1, Type: element.WAY, Role: "outer", Way: &w1},
|
||||||
{2, element.WAY, "inner", &w2},
|
{Id: 2, Type: element.WAY, Role: "inner", Way: &w2},
|
||||||
}
|
}
|
||||||
|
|
||||||
geom, err := buildRelation(&rel, 3857)
|
geom, err := buildRelation(&rel, 3857)
|
||||||
|
@ -150,9 +149,9 @@ func TestMultiPolygonWithMultipleHoles(t *testing.T) {
|
||||||
rel := element.Relation{
|
rel := element.Relation{
|
||||||
OSMElem: element.OSMElem{Id: 1, Tags: element.Tags{"landusage": "forest"}}}
|
OSMElem: element.OSMElem{Id: 1, Tags: element.Tags{"landusage": "forest"}}}
|
||||||
rel.Members = []element.Member{
|
rel.Members = []element.Member{
|
||||||
{1, element.WAY, "outer", &w1},
|
{Id: 1, Type: element.WAY, Role: "outer", Way: &w1},
|
||||||
{2, element.WAY, "inner", &w2},
|
{Id: 2, Type: element.WAY, Role: "inner", Way: &w2},
|
||||||
{3, element.WAY, "inner", &w3},
|
{Id: 3, Type: element.WAY, Role: "inner", Way: &w3},
|
||||||
}
|
}
|
||||||
|
|
||||||
geom, err := buildRelation(&rel, 3857)
|
geom, err := buildRelation(&rel, 3857)
|
||||||
|
@ -217,11 +216,11 @@ func TestMultiPolygonWithNeastedHoles(t *testing.T) {
|
||||||
|
|
||||||
rel := element.Relation{OSMElem: element.OSMElem{Id: 1}}
|
rel := element.Relation{OSMElem: element.OSMElem{Id: 1}}
|
||||||
rel.Members = []element.Member{
|
rel.Members = []element.Member{
|
||||||
{1, element.WAY, "outer", &w1},
|
{Id: 1, Type: element.WAY, Role: "outer", Way: &w1},
|
||||||
{2, element.WAY, "inner", &w2},
|
{Id: 2, Type: element.WAY, Role: "inner", Way: &w2},
|
||||||
{3, element.WAY, "inner", &w3},
|
{Id: 3, Type: element.WAY, Role: "inner", Way: &w3},
|
||||||
{4, element.WAY, "inner", &w4},
|
{Id: 4, Type: element.WAY, Role: "inner", Way: &w4},
|
||||||
{5, element.WAY, "inner", &w5},
|
{Id: 5, Type: element.WAY, Role: "inner", Way: &w5},
|
||||||
}
|
}
|
||||||
|
|
||||||
geom, err := buildRelation(&rel, 3857)
|
geom, err := buildRelation(&rel, 3857)
|
||||||
|
@ -264,9 +263,9 @@ func TestPolygonFromThreeWays(t *testing.T) {
|
||||||
|
|
||||||
rel := element.Relation{OSMElem: element.OSMElem{Id: 1}}
|
rel := element.Relation{OSMElem: element.OSMElem{Id: 1}}
|
||||||
rel.Members = []element.Member{
|
rel.Members = []element.Member{
|
||||||
{1, element.WAY, "outer", &w1},
|
{Id: 1, Type: element.WAY, Role: "outer", Way: &w1},
|
||||||
{2, element.WAY, "inner", &w2},
|
{Id: 2, Type: element.WAY, Role: "inner", Way: &w2},
|
||||||
{3, element.WAY, "inner", &w3},
|
{Id: 3, Type: element.WAY, Role: "inner", Way: &w3},
|
||||||
}
|
}
|
||||||
|
|
||||||
geom, err := buildRelation(&rel, 3857)
|
geom, err := buildRelation(&rel, 3857)
|
||||||
|
@ -317,9 +316,9 @@ func TestTouchingPolygonsWithHole(t *testing.T) {
|
||||||
|
|
||||||
rel := element.Relation{OSMElem: element.OSMElem{Id: 1, Tags: element.Tags{"water": "riverbank"}}}
|
rel := element.Relation{OSMElem: element.OSMElem{Id: 1, Tags: element.Tags{"water": "riverbank"}}}
|
||||||
rel.Members = []element.Member{
|
rel.Members = []element.Member{
|
||||||
{1, element.WAY, "outer", &w1},
|
{Id: 1, Type: element.WAY, Role: "outer", Way: &w1},
|
||||||
{2, element.WAY, "outer", &w2},
|
{Id: 2, Type: element.WAY, Role: "outer", Way: &w2},
|
||||||
{3, element.WAY, "inner", &w3},
|
{Id: 3, Type: element.WAY, Role: "inner", Way: &w3},
|
||||||
}
|
}
|
||||||
geom, err := buildRelation(&rel, 3857)
|
geom, err := buildRelation(&rel, 3857)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
@ -359,8 +358,8 @@ func TestInsertedWaysDifferentTags(t *testing.T) {
|
||||||
|
|
||||||
rel := element.Relation{OSMElem: element.OSMElem{Id: 1, Tags: element.Tags{"landusage": "forest"}}}
|
rel := element.Relation{OSMElem: element.OSMElem{Id: 1, Tags: element.Tags{"landusage": "forest"}}}
|
||||||
rel.Members = []element.Member{
|
rel.Members = []element.Member{
|
||||||
{1, element.WAY, "outer", &w1},
|
{Id: 1, Type: element.WAY, Role: "outer", Way: &w1},
|
||||||
{2, element.WAY, "inner", &w2},
|
{Id: 2, Type: element.WAY, Role: "inner", Way: &w2},
|
||||||
}
|
}
|
||||||
|
|
||||||
geom, err := buildRelation(&rel, 3857)
|
geom, err := buildRelation(&rel, 3857)
|
||||||
|
@ -401,8 +400,8 @@ func TestInsertMultipleTags(t *testing.T) {
|
||||||
|
|
||||||
rel := element.Relation{OSMElem: element.OSMElem{Id: 1, Tags: element.Tags{"landusage": "forest"}}}
|
rel := element.Relation{OSMElem: element.OSMElem{Id: 1, Tags: element.Tags{"landusage": "forest"}}}
|
||||||
rel.Members = []element.Member{
|
rel.Members = []element.Member{
|
||||||
{1, element.WAY, "outer", &w1}, // also highway=secondary
|
{Id: 1, Type: element.WAY, Role: "outer", Way: &w1}, // also highway=secondary
|
||||||
{2, element.WAY, "inner", &w2},
|
{Id: 2, Type: element.WAY, Role: "inner", Way: &w2},
|
||||||
}
|
}
|
||||||
|
|
||||||
geom, err := buildRelation(&rel, 3857)
|
geom, err := buildRelation(&rel, 3857)
|
||||||
|
@ -450,8 +449,8 @@ func TestBrokenPolygonSelfIntersect(t *testing.T) {
|
||||||
|
|
||||||
rel1 := element.Relation{OSMElem: element.OSMElem{Id: 1}}
|
rel1 := element.Relation{OSMElem: element.OSMElem{Id: 1}}
|
||||||
rel1.Members = []element.Member{
|
rel1.Members = []element.Member{
|
||||||
{1, element.WAY, "outer", &w1},
|
{Id: 1, Type: element.WAY, Role: "outer", Way: &w1},
|
||||||
{2, element.WAY, "inner", &w2},
|
{Id: 2, Type: element.WAY, Role: "inner", Way: &w2},
|
||||||
}
|
}
|
||||||
|
|
||||||
geom1, err := buildRelation(&rel1, 3857)
|
geom1, err := buildRelation(&rel1, 3857)
|
||||||
|
@ -491,8 +490,8 @@ func TestBrokenPolygonSelfIntersect(t *testing.T) {
|
||||||
|
|
||||||
rel2 := element.Relation{OSMElem: element.OSMElem{Id: 1}}
|
rel2 := element.Relation{OSMElem: element.OSMElem{Id: 1}}
|
||||||
rel2.Members = []element.Member{
|
rel2.Members = []element.Member{
|
||||||
{1, element.WAY, "outer", &w3},
|
{Id: 1, Type: element.WAY, Role: "outer", Way: &w3},
|
||||||
{2, element.WAY, "inner", &w2},
|
{Id: 2, Type: element.WAY, Role: "inner", Way: &w2},
|
||||||
}
|
}
|
||||||
|
|
||||||
geom2, err := buildRelation(&rel2, 3857)
|
geom2, err := buildRelation(&rel2, 3857)
|
||||||
|
@ -540,8 +539,8 @@ func TestBrokenPolygonSelfIntersectTriangle(t *testing.T) {
|
||||||
|
|
||||||
rel := element.Relation{OSMElem: element.OSMElem{Id: 1}}
|
rel := element.Relation{OSMElem: element.OSMElem{Id: 1}}
|
||||||
rel.Members = []element.Member{
|
rel.Members = []element.Member{
|
||||||
{1, element.WAY, "outer", &w1},
|
{Id: 1, Type: element.WAY, Role: "outer", Way: &w1},
|
||||||
{2, element.WAY, "inner", &w2},
|
{Id: 2, Type: element.WAY, Role: "inner", Way: &w2},
|
||||||
}
|
}
|
||||||
|
|
||||||
geom, err := buildRelation(&rel, 3857)
|
geom, err := buildRelation(&rel, 3857)
|
||||||
|
@ -580,8 +579,8 @@ func TestBrokenPolygonSelfIntersectTriangle(t *testing.T) {
|
||||||
|
|
||||||
rel = element.Relation{OSMElem: element.OSMElem{Id: 1}}
|
rel = element.Relation{OSMElem: element.OSMElem{Id: 1}}
|
||||||
rel.Members = []element.Member{
|
rel.Members = []element.Member{
|
||||||
{1, element.WAY, "outer", &w3},
|
{Id: 1, Type: element.WAY, Role: "outer", Way: &w3},
|
||||||
{2, element.WAY, "inner", &w4},
|
{Id: 2, Type: element.WAY, Role: "inner", Way: &w4},
|
||||||
}
|
}
|
||||||
|
|
||||||
geom, err = buildRelation(&rel, 3857)
|
geom, err = buildRelation(&rel, 3857)
|
||||||
|
@ -599,3 +598,63 @@ func TestBrokenPolygonSelfIntersectTriangle(t *testing.T) {
|
||||||
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestOpenRing(t *testing.T) {
|
||||||
|
w1 := makeWay(1, element.Tags{}, []coord{
|
||||||
|
{1, 0, 0},
|
||||||
|
{2, 10, 0},
|
||||||
|
{3, 10, 10},
|
||||||
|
{4, 0, 10},
|
||||||
|
})
|
||||||
|
|
||||||
|
rel := element.Relation{
|
||||||
|
OSMElem: element.OSMElem{Id: 1, Tags: element.Tags{}}}
|
||||||
|
rel.Members = []element.Member{
|
||||||
|
{Id: 1, Type: element.WAY, Role: "outer", Way: &w1},
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err := buildRelation(&rel, 3857)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("no error from open ring")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestClosedAndOpenRing(t *testing.T) {
|
||||||
|
w1 := makeWay(1, element.Tags{}, []coord{
|
||||||
|
{1, 0, 0},
|
||||||
|
{2, 10, 0},
|
||||||
|
{3, 10, 10},
|
||||||
|
{4, 0, 10},
|
||||||
|
{1, 0, 0},
|
||||||
|
})
|
||||||
|
w2 := makeWay(2, element.Tags{}, []coord{
|
||||||
|
{5, 0, 0},
|
||||||
|
{6, -5, -2},
|
||||||
|
})
|
||||||
|
rel := element.Relation{
|
||||||
|
OSMElem: element.OSMElem{Id: 1, Tags: element.Tags{}}}
|
||||||
|
rel.Members = []element.Member{
|
||||||
|
{Id: 1, Type: element.WAY, Role: "outer", Way: &w1},
|
||||||
|
{Id: 2, Type: element.WAY, Role: "outer", Way: &w2},
|
||||||
|
}
|
||||||
|
|
||||||
|
prep, err := PrepareRelation(&rel, 3857, 0.1)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// open ring is excluded
|
||||||
|
if len(prep.rings) != 1 {
|
||||||
|
t.Fatal("expected single ring")
|
||||||
|
}
|
||||||
|
geom, err := prep.Build()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
g := geos.NewGeos()
|
||||||
|
defer g.Finish()
|
||||||
|
|
||||||
|
if !g.IsValid(geom.Geom) {
|
||||||
|
t.Fatal("geometry not valid", g.AsWkt(geom.Geom))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
|
@ -182,6 +182,8 @@ func Import() {
|
||||||
relations,
|
relations,
|
||||||
db, progress,
|
db, progress,
|
||||||
tagmapping.PolygonMatcher(),
|
tagmapping.PolygonMatcher(),
|
||||||
|
tagmapping.RelationMatcher(),
|
||||||
|
tagmapping.RelationMemberMatcher(),
|
||||||
config.BaseOptions.Srid)
|
config.BaseOptions.Srid)
|
||||||
relWriter.SetLimiter(geometryLimiter)
|
relWriter.SetLimiter(geometryLimiter)
|
||||||
relWriter.EnableConcurrent()
|
relWriter.EnableConcurrent()
|
||||||
|
|
|
@ -11,11 +11,12 @@ import (
|
||||||
)
|
)
|
||||||
|
|
||||||
type Field struct {
|
type Field struct {
|
||||||
Name string `yaml:"name"`
|
Name string `yaml:"name"`
|
||||||
Key Key `yaml:"key"`
|
Key Key `yaml:"key"`
|
||||||
Keys []Key `yaml:"keys"`
|
Keys []Key `yaml:"keys"`
|
||||||
Type string `yaml:"type"`
|
Type string `yaml:"type"`
|
||||||
Args map[string]interface{} `yaml:"args"`
|
Args map[string]interface{} `yaml:"args"`
|
||||||
|
FromMember bool `yaml:"from_member"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type Table struct {
|
type Table struct {
|
||||||
|
@ -133,6 +134,10 @@ func (tt *TableType) UnmarshalJSON(data []byte) error {
|
||||||
*tt = PolygonTable
|
*tt = PolygonTable
|
||||||
case `"geometry"`:
|
case `"geometry"`:
|
||||||
*tt = GeometryTable
|
*tt = GeometryTable
|
||||||
|
case `"relation"`:
|
||||||
|
*tt = RelationTable
|
||||||
|
case `"relation_member"`:
|
||||||
|
*tt = RelationMemberTable
|
||||||
default:
|
default:
|
||||||
return errors.New("unknown type " + string(data))
|
return errors.New("unknown type " + string(data))
|
||||||
}
|
}
|
||||||
|
@ -140,10 +145,12 @@ func (tt *TableType) UnmarshalJSON(data []byte) error {
|
||||||
}
|
}
|
||||||
|
|
||||||
const (
|
const (
|
||||||
PolygonTable TableType = "polygon"
|
PolygonTable TableType = "polygon"
|
||||||
LineStringTable TableType = "linestring"
|
LineStringTable TableType = "linestring"
|
||||||
PointTable TableType = "point"
|
PointTable TableType = "point"
|
||||||
GeometryTable TableType = "geometry"
|
GeometryTable TableType = "geometry"
|
||||||
|
RelationTable TableType = "relation"
|
||||||
|
RelationMemberTable TableType = "relation_member"
|
||||||
)
|
)
|
||||||
|
|
||||||
func NewMapping(filename string) (*Mapping, error) {
|
func NewMapping(filename string) (*Mapping, error) {
|
||||||
|
@ -242,7 +249,7 @@ func (m *Mapping) tables(tableType TableType) map[string]*TableFields {
|
||||||
|
|
||||||
func (m *Mapping) extraTags(tableType TableType, tags map[Key]bool) {
|
func (m *Mapping) extraTags(tableType TableType, tags map[Key]bool) {
|
||||||
for _, t := range m.Tables {
|
for _, t := range m.Tables {
|
||||||
if t.Type != tableType {
|
if t.Type != tableType && t.Type != "geometry" {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
for key, _ := range t.ExtraTags() {
|
for key, _ := range t.ExtraTags() {
|
||||||
|
|
|
@ -17,26 +17,31 @@ var AvailableFieldTypes map[string]FieldType
|
||||||
|
|
||||||
func init() {
|
func init() {
|
||||||
AvailableFieldTypes = map[string]FieldType{
|
AvailableFieldTypes = map[string]FieldType{
|
||||||
"bool": {"bool", "bool", Bool, nil},
|
"bool": {"bool", "bool", Bool, nil, nil, false},
|
||||||
"boolint": {"boolint", "int8", BoolInt, nil},
|
"boolint": {"boolint", "int8", BoolInt, nil, nil, false},
|
||||||
"id": {"id", "int64", Id, nil},
|
"id": {"id", "int64", Id, nil, nil, false},
|
||||||
"string": {"string", "string", String, nil},
|
"string": {"string", "string", String, nil, nil, false},
|
||||||
"direction": {"direction", "int8", Direction, nil},
|
"direction": {"direction", "int8", Direction, nil, nil, false},
|
||||||
"integer": {"integer", "int32", Integer, nil},
|
"integer": {"integer", "int32", Integer, nil, nil, false},
|
||||||
"mapping_key": {"mapping_key", "string", KeyName, nil},
|
"mapping_key": {"mapping_key", "string", KeyName, nil, nil, false},
|
||||||
"mapping_value": {"mapping_value", "string", ValueName, nil},
|
"mapping_value": {"mapping_value", "string", ValueName, nil, nil, false},
|
||||||
"geometry": {"geometry", "geometry", Geometry, nil},
|
"member_id": {"member_id", "int64", nil, nil, RelationMemberID, true},
|
||||||
"validated_geometry": {"validated_geometry", "validated_geometry", Geometry, nil},
|
"member_role": {"member_role", "string", nil, nil, RelationMemberRole, true},
|
||||||
"hstore_tags": {"hstore_tags", "hstore_string", HstoreString, nil},
|
"member_type": {"member_type", "int8", nil, nil, RelationMemberType, true},
|
||||||
"wayzorder": {"wayzorder", "int32", WayZOrder, nil},
|
"member_index": {"member_index", "int32", nil, nil, RelationMemberIndex, true},
|
||||||
"pseudoarea": {"pseudoarea", "float32", PseudoArea, nil},
|
"geometry": {"geometry", "geometry", Geometry, nil, nil, false},
|
||||||
"zorder": {"zorder", "int32", nil, MakeZOrder},
|
"validated_geometry": {"validated_geometry", "validated_geometry", Geometry, nil, nil, false},
|
||||||
"enumerate": {"enumerate", "int32", nil, MakeEnumerate},
|
"hstore_tags": {"hstore_tags", "hstore_string", HstoreString, nil, nil, false},
|
||||||
"string_suffixreplace": {"string_suffixreplace", "string", nil, MakeSuffixReplace},
|
"wayzorder": {"wayzorder", "int32", WayZOrder, nil, nil, false},
|
||||||
|
"pseudoarea": {"pseudoarea", "float32", PseudoArea, nil, nil, false},
|
||||||
|
"zorder": {"zorder", "int32", nil, MakeZOrder, nil, false},
|
||||||
|
"enumerate": {"enumerate", "int32", nil, MakeEnumerate, nil, false},
|
||||||
|
"string_suffixreplace": {"string_suffixreplace", "string", nil, MakeSuffixReplace, nil, false},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
type MakeValue func(string, *element.OSMElem, *geom.Geometry, Match) interface{}
|
type MakeValue func(string, *element.OSMElem, *geom.Geometry, Match) interface{}
|
||||||
|
type MakeMemberValue func(*element.Relation, *element.Member, Match) interface{}
|
||||||
|
|
||||||
type MakeMakeValue func(string, FieldType, Field) (MakeValue, error)
|
type MakeMakeValue func(string, FieldType, Field) (MakeValue, error)
|
||||||
|
|
||||||
|
@ -55,6 +60,22 @@ func (f *FieldSpec) Value(elem *element.OSMElem, geom *geom.Geometry, match Matc
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (f *FieldSpec) MemberValue(rel *element.Relation, member *element.Member, geom *geom.Geometry, match Match) interface{} {
|
||||||
|
if f.Type.Func != nil {
|
||||||
|
if f.Type.FromMember {
|
||||||
|
if member.Elem == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return f.Type.Func(member.Elem.Tags[string(f.Key)], member.Elem, geom, match)
|
||||||
|
}
|
||||||
|
return f.Type.Func(rel.Tags[string(f.Key)], &rel.OSMElem, geom, match)
|
||||||
|
}
|
||||||
|
if f.Type.MemberFunc != nil {
|
||||||
|
return f.Type.MemberFunc(rel, member, match)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
type TableFields struct {
|
type TableFields struct {
|
||||||
fields []FieldSpec
|
fields []FieldSpec
|
||||||
}
|
}
|
||||||
|
@ -67,6 +88,14 @@ func (t *TableFields) MakeRow(elem *element.OSMElem, geom *geom.Geometry, match
|
||||||
return row
|
return row
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (t *TableFields) MakeMemberRow(rel *element.Relation, member *element.Member, geom *geom.Geometry, match Match) []interface{} {
|
||||||
|
var row []interface{}
|
||||||
|
for _, field := range t.fields {
|
||||||
|
row = append(row, field.MemberValue(rel, member, geom, match))
|
||||||
|
}
|
||||||
|
return row
|
||||||
|
}
|
||||||
|
|
||||||
func (field *Field) FieldType() *FieldType {
|
func (field *Field) FieldType() *FieldType {
|
||||||
if fieldType, ok := AvailableFieldTypes[field.Type]; ok {
|
if fieldType, ok := AvailableFieldTypes[field.Type]; ok {
|
||||||
if fieldType.MakeFunc != nil {
|
if fieldType.MakeFunc != nil {
|
||||||
|
@ -75,8 +104,9 @@ func (field *Field) FieldType() *FieldType {
|
||||||
log.Print(err)
|
log.Print(err)
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
fieldType = FieldType{fieldType.Name, fieldType.GoType, makeValue, nil}
|
fieldType = FieldType{fieldType.Name, fieldType.GoType, makeValue, nil, nil, fieldType.FromMember}
|
||||||
}
|
}
|
||||||
|
fieldType.FromMember = field.FromMember
|
||||||
return &fieldType
|
return &fieldType
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
|
@ -101,10 +131,12 @@ func (t *Table) TableFields() *TableFields {
|
||||||
}
|
}
|
||||||
|
|
||||||
type FieldType struct {
|
type FieldType struct {
|
||||||
Name string
|
Name string
|
||||||
GoType string
|
GoType string
|
||||||
Func MakeValue
|
Func MakeValue
|
||||||
MakeFunc MakeMakeValue
|
MakeFunc MakeMakeValue
|
||||||
|
MemberFunc MakeMemberValue
|
||||||
|
FromMember bool
|
||||||
}
|
}
|
||||||
|
|
||||||
func Bool(val string, elem *element.OSMElem, geom *geom.Geometry, match Match) interface{} {
|
func Bool(val string, elem *element.OSMElem, geom *geom.Geometry, match Match) interface{} {
|
||||||
|
@ -145,6 +177,27 @@ func ValueName(val string, elem *element.OSMElem, geom *geom.Geometry, match Mat
|
||||||
return match.Value
|
return match.Value
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func RelationMemberType(rel *element.Relation, member *element.Member, match Match) interface{} {
|
||||||
|
return member.Type
|
||||||
|
}
|
||||||
|
|
||||||
|
func RelationMemberRole(rel *element.Relation, member *element.Member, match Match) interface{} {
|
||||||
|
return member.Role
|
||||||
|
}
|
||||||
|
|
||||||
|
func RelationMemberID(rel *element.Relation, member *element.Member, match Match) interface{} {
|
||||||
|
return member.Id
|
||||||
|
}
|
||||||
|
|
||||||
|
func RelationMemberIndex(rel *element.Relation, member *element.Member, match Match) interface{} {
|
||||||
|
for i := range rel.Members {
|
||||||
|
if rel.Members[i].Id == member.Id {
|
||||||
|
return i
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
|
||||||
func Direction(val string, elem *element.OSMElem, geom *geom.Geometry, match Match) interface{} {
|
func Direction(val string, elem *element.OSMElem, geom *geom.Geometry, match Match) interface{} {
|
||||||
if val == "1" || val == "yes" || val == "true" {
|
if val == "1" || val == "yes" || val == "true" {
|
||||||
return 1
|
return 1
|
||||||
|
|
|
@ -7,23 +7,37 @@ import (
|
||||||
|
|
||||||
func (m *Mapping) PointMatcher() NodeMatcher {
|
func (m *Mapping) PointMatcher() NodeMatcher {
|
||||||
mappings := make(TagTables)
|
mappings := make(TagTables)
|
||||||
m.mappings("point", mappings)
|
m.mappings(PointTable, mappings)
|
||||||
filters := m.ElementFilters()
|
filters := m.ElementFilters()
|
||||||
return &tagMatcher{mappings, m.tables("point"), filters, false}
|
return &tagMatcher{mappings, m.tables(PointTable), filters, false}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (m *Mapping) LineStringMatcher() WayMatcher {
|
func (m *Mapping) LineStringMatcher() WayMatcher {
|
||||||
mappings := make(TagTables)
|
mappings := make(TagTables)
|
||||||
m.mappings("linestring", mappings)
|
m.mappings(LineStringTable, mappings)
|
||||||
filters := m.ElementFilters()
|
filters := m.ElementFilters()
|
||||||
return &tagMatcher{mappings, m.tables("linestring"), filters, false}
|
return &tagMatcher{mappings, m.tables(LineStringTable), filters, false}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (m *Mapping) PolygonMatcher() RelWayMatcher {
|
func (m *Mapping) PolygonMatcher() RelWayMatcher {
|
||||||
mappings := make(TagTables)
|
mappings := make(TagTables)
|
||||||
m.mappings("polygon", mappings)
|
m.mappings(PolygonTable, mappings)
|
||||||
filters := m.ElementFilters()
|
filters := m.ElementFilters()
|
||||||
return &tagMatcher{mappings, m.tables("polygon"), filters, true}
|
return &tagMatcher{mappings, m.tables(PolygonTable), filters, true}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *Mapping) RelationMatcher() RelationMatcher {
|
||||||
|
mappings := make(TagTables)
|
||||||
|
m.mappings(RelationTable, mappings)
|
||||||
|
filters := m.ElementFilters()
|
||||||
|
return &tagMatcher{mappings, m.tables(RelationTable), filters, true}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *Mapping) RelationMemberMatcher() RelationMatcher {
|
||||||
|
mappings := make(TagTables)
|
||||||
|
m.mappings(RelationMemberTable, mappings)
|
||||||
|
filters := m.ElementFilters()
|
||||||
|
return &tagMatcher{mappings, m.tables(RelationMemberTable), filters, true}
|
||||||
}
|
}
|
||||||
|
|
||||||
type Match struct {
|
type Match struct {
|
||||||
|
@ -61,6 +75,10 @@ func (m *Match) Row(elem *element.OSMElem, geom *geom.Geometry) []interface{} {
|
||||||
return m.tableFields.MakeRow(elem, geom, *m)
|
return m.tableFields.MakeRow(elem, geom, *m)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (m *Match) MemberRow(rel *element.Relation, member *element.Member, geom *geom.Geometry) []interface{} {
|
||||||
|
return m.tableFields.MakeMemberRow(rel, member, geom, *m)
|
||||||
|
}
|
||||||
|
|
||||||
func (tm *tagMatcher) MatchNode(node *element.Node) []Match {
|
func (tm *tagMatcher) MatchNode(node *element.Node) []Match {
|
||||||
return tm.match(&node.Tags)
|
return tm.match(&node.Tags)
|
||||||
}
|
}
|
||||||
|
|
|
@ -7,7 +7,7 @@ import (
|
||||||
)
|
)
|
||||||
|
|
||||||
func BenchmarkTagMatch(b *testing.B) {
|
func BenchmarkTagMatch(b *testing.B) {
|
||||||
m, err := NewMapping("matcher_test_mapping.yml")
|
m, err := NewMapping("test_mapping.yml")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
b.Fatal(err)
|
b.Fatal(err)
|
||||||
}
|
}
|
||||||
|
|
|
@ -3,7 +3,6 @@ package pbf
|
||||||
import (
|
import (
|
||||||
"bytes"
|
"bytes"
|
||||||
"compress/zlib"
|
"compress/zlib"
|
||||||
"fmt"
|
|
||||||
"io"
|
"io"
|
||||||
"log"
|
"log"
|
||||||
"os"
|
"os"
|
||||||
|
@ -15,13 +14,12 @@ import (
|
||||||
|
|
||||||
func BenchmarkHello(b *testing.B) {
|
func BenchmarkHello(b *testing.B) {
|
||||||
b.StopTimer()
|
b.StopTimer()
|
||||||
pbf, err := Open("../azores.osm.pbf")
|
pbf, err := Open("./monaco-20150428.osm.pbf")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
panic(err)
|
panic(err)
|
||||||
}
|
}
|
||||||
|
|
||||||
for pos := range pbf.BlockPositions() {
|
for pos := range pbf.BlockPositions() {
|
||||||
fmt.Println(pos.size, pos.offset)
|
|
||||||
b.StartTimer()
|
b.StartTimer()
|
||||||
for i := 0; i < b.N; i++ {
|
for i := 0; i < b.N; i++ {
|
||||||
readPrimitiveBlock(pos)
|
readPrimitiveBlock(pos)
|
||||||
|
@ -43,7 +41,7 @@ func BenchmarkHello(b *testing.B) {
|
||||||
func BenchmarkPrimitiveBlock(b *testing.B) {
|
func BenchmarkPrimitiveBlock(b *testing.B) {
|
||||||
b.StopTimer()
|
b.StopTimer()
|
||||||
|
|
||||||
file, err := os.Open("../azores.osm.pbf")
|
file, err := os.Open("./monaco-20150428.osm.pbf")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
log.Panic(err)
|
log.Panic(err)
|
||||||
}
|
}
|
||||||
|
@ -52,8 +50,8 @@ func BenchmarkPrimitiveBlock(b *testing.B) {
|
||||||
var block = &osmpbf.PrimitiveBlock{}
|
var block = &osmpbf.PrimitiveBlock{}
|
||||||
var blob = &osmpbf.Blob{}
|
var blob = &osmpbf.Blob{}
|
||||||
|
|
||||||
var size = 56092
|
var size = 79566
|
||||||
var offset int64 = 197
|
var offset int64 = 155
|
||||||
|
|
||||||
blobData := make([]byte, size)
|
blobData := make([]byte, size)
|
||||||
file.Seek(offset, 0)
|
file.Seek(offset, 0)
|
||||||
|
|
|
@ -1,37 +1,31 @@
|
||||||
.PHONY: build all test clean
|
.PHONY: all test clean files
|
||||||
|
|
||||||
IMPOSM_BIN=../imposm3
|
|
||||||
|
|
||||||
ifdef VERBOSE
|
ifdef VERBOSE
|
||||||
NOSEOPTS = -vs
|
TESTOPTS = -v
|
||||||
else
|
else
|
||||||
NOSEOPTS = -v
|
TESTOPTS =
|
||||||
endif
|
endif
|
||||||
|
|
||||||
all: build test
|
all: test
|
||||||
|
|
||||||
build:
|
|
||||||
cd ..; make build
|
|
||||||
|
|
||||||
$(IMPOSM_BIN): build
|
|
||||||
|
|
||||||
clean:
|
clean:
|
||||||
rm -rf build
|
rm -rf build
|
||||||
|
|
||||||
|
PBF_FILES=$(addprefix build/,$(patsubst %.osm,%.pbf,$(wildcard *.osm)))
|
||||||
|
OSCGZ_FILES=$(addprefix build/,$(patsubst %.osc,%.osc.gz,$(wildcard *.osc)))
|
||||||
|
|
||||||
build/%.pbf: %.osm
|
build/%.pbf: %.osm
|
||||||
@mkdir -p build
|
@mkdir -p build
|
||||||
osmosis --read-xml $< --write-pbf $@ omitmetadata=true
|
osmosis --read-xml $< --sort type="TypeThenId" --write-pbf $@ omitmetadata=true
|
||||||
|
|
||||||
build/%.osc.gz: %.osc
|
build/%.osc.gz: %.osc
|
||||||
@mkdir -p build
|
@mkdir -p build
|
||||||
gzip --stdout $< > $@
|
gzip --stdout $< > $@
|
||||||
|
|
||||||
test: .lasttestrun_complete_db .lasttestrun_single_table
|
files: $(PBF_FILES) $(OSCGZ_FILES)
|
||||||
|
|
||||||
.lasttestrun_complete_db: $(IMPOSM_BIN) complete_db_test.py build/complete_db.osc.gz build/complete_db.pbf
|
test: files
|
||||||
nosetests complete_db_test.py $(NOSEOPTS)
|
(cd .. && godep go test ./test $(TESTOPTS))
|
||||||
@touch .lasttestrun_complete_db
|
|
||||||
|
|
||||||
.lasttestrun_single_table: $(IMPOSM_BIN) single_table_test.py build/single_table.osc.gz build/single_table.pbf
|
route_relation: files
|
||||||
nosetests single_table_test.py $(NOSEOPTS)
|
(cd .. && godep go test -test.run TestRouteRelation_ ./test $(TESTOPTS))
|
||||||
@touch .lasttestrun_single_table
|
|
||||||
|
|
|
@ -190,9 +190,9 @@
|
||||||
<create>
|
<create>
|
||||||
<node id="201001" version="1" timestamp="2011-11-11T00:11:11Z" lat="32" lon="10"/>
|
<node id="201001" version="1" timestamp="2011-11-11T00:11:11Z" lat="32" lon="10"/>
|
||||||
<node id="201002" version="1" timestamp="2011-11-11T00:11:11Z" lat="32" lon="11"/>
|
<node id="201002" version="1" timestamp="2011-11-11T00:11:11Z" lat="32" lon="11"/>
|
||||||
<node id="201003" version="1" timestamp="2011-11-11T00:11:11Z" lat="34" lon="10"/>
|
<node id="201003" version="1" timestamp="2011-11-11T00:11:11Z" lat="34" lon="11"/>
|
||||||
<node id="201004" version="1" timestamp="2011-11-11T00:11:11Z" lat="34" lon="11"/>
|
<node id="201004" version="1" timestamp="2011-11-11T00:11:11Z" lat="34" lon="10"/>
|
||||||
<way id="201001" version="1" timestamp="2011-11-11T00:11:11Z">
|
<way id="201051" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
<nd ref="201001"/>
|
<nd ref="201001"/>
|
||||||
<nd ref="201002"/>
|
<nd ref="201002"/>
|
||||||
<nd ref="201003"/>
|
<nd ref="201003"/>
|
||||||
|
@ -200,8 +200,8 @@
|
||||||
<nd ref="201001"/>
|
<nd ref="201001"/>
|
||||||
<tag k="highway" v="residential"/>
|
<tag k="highway" v="residential"/>
|
||||||
</way>
|
</way>
|
||||||
<relation id="201001" version="1" timestamp="2011-11-11T00:11:11Z">
|
<relation id="201091" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
<member type="way" ref="201001" role="outer"/>
|
<member type="way" ref="201051" role="outer"/>
|
||||||
<tag k="type" v="multipolygon"/>
|
<tag k="type" v="multipolygon"/>
|
||||||
<tag k="landuse" v="park"/>
|
<tag k="landuse" v="park"/>
|
||||||
</relation>
|
</relation>
|
||||||
|
@ -210,11 +210,26 @@
|
||||||
<!-- test modification of one relation (201102) does not duplicate
|
<!-- test modification of one relation (201102) does not duplicate
|
||||||
relation (201101) with shared way (checks #65) -->
|
relation (201101) with shared way (checks #65) -->
|
||||||
<modify>
|
<modify>
|
||||||
<relation id="201102" version="1" timestamp="2011-11-11T00:11:11Z">
|
<relation id="201192" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
<member type="way" ref="201101" role="outer"/>
|
<member type="way" ref="201151" role="outer"/>
|
||||||
<tag k="type" v="multipolygon"/>
|
<tag k="type" v="multipolygon"/>
|
||||||
<tag k="landuse" v="forest"/>
|
<tag k="landuse" v="forest"/>
|
||||||
</relation>
|
</relation>
|
||||||
</modify>
|
</modify>
|
||||||
|
|
||||||
|
<!-- test that relations with unsupported types are not inserted with updates -->
|
||||||
|
<create>
|
||||||
|
<relation id="201291" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
|
<member type="way" ref="201251" role="platform"/>
|
||||||
|
<tag k="type" v="unmapped_relation_type"/>
|
||||||
|
</relation>
|
||||||
|
</create>
|
||||||
|
|
||||||
|
|
||||||
|
<!-- test that relations are updated after modified node -->
|
||||||
|
<modify>
|
||||||
|
<node id="52101" version="1" timestamp="2011-11-11T00:11:11Z" lat="61" lon="10"/>
|
||||||
|
</modify>
|
||||||
|
|
||||||
|
|
||||||
</osmChange>
|
</osmChange>
|
||||||
|
|
|
@ -386,6 +386,31 @@
|
||||||
<tag k="landuse" v="park"/>
|
<tag k="landuse" v="park"/>
|
||||||
</way>
|
</way>
|
||||||
|
|
||||||
|
<!-- relation/way with "gap" (ways overlap, but are only sharing one endpoint) -->
|
||||||
|
<node id="7401" version="1" timestamp="2011-11-11T00:11:11Z" lat="60" lon="60"/>
|
||||||
|
<node id="7402" version="1" timestamp="2011-11-11T00:11:11Z" lat="60" lon="62"/>
|
||||||
|
<node id="7403" version="1" timestamp="2011-11-11T00:11:11Z" lat="62" lon="62"/>
|
||||||
|
<node id="7404" version="1" timestamp="2011-11-11T00:11:11Z" lat="62" lon="60"/>
|
||||||
|
<way id="7401" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
|
<nd ref="7401"/>
|
||||||
|
<nd ref="7402"/>
|
||||||
|
<nd ref="7403"/>
|
||||||
|
<nd ref="7404"/>
|
||||||
|
<nd ref="7401"/>
|
||||||
|
</way>
|
||||||
|
<way id="7402" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
|
<!-- connected to other way, but not at start/end node and neither a ring on its own -->
|
||||||
|
<nd ref="7402"/>
|
||||||
|
<nd ref="7404"/>
|
||||||
|
</way>
|
||||||
|
<relation id="7401" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
|
<member type="way" ref="7401" role="outer"/>
|
||||||
|
<member type="way" ref="7402" role="outer"/>
|
||||||
|
<tag k="name" v="rel 7401"/>
|
||||||
|
<tag k="landuse" v="park"/>
|
||||||
|
<tag k="type" v="multipolygon"/>
|
||||||
|
</relation>
|
||||||
|
|
||||||
<!-- test that single node ways or incomplete polygons are _not_ inserted -->
|
<!-- test that single node ways or incomplete polygons are _not_ inserted -->
|
||||||
<node id="30001" version="1" timestamp="2011-11-11T00:11:11Z" lat="47" lon="80"/>
|
<node id="30001" version="1" timestamp="2011-11-11T00:11:11Z" lat="47" lon="80"/>
|
||||||
<node id="30002" version="1" timestamp="2011-11-11T00:11:11Z" lat="47" lon="80"/>
|
<node id="30002" version="1" timestamp="2011-11-11T00:11:11Z" lat="47" lon="80"/>
|
||||||
|
@ -994,6 +1019,26 @@
|
||||||
<tag k="building" v="mp"/>
|
<tag k="building" v="mp"/>
|
||||||
</relation>
|
</relation>
|
||||||
|
|
||||||
|
|
||||||
|
<!-- test that relations are updated after modified node -->
|
||||||
|
<node id="52101" version="1" timestamp="2011-11-11T00:11:11Z" lat="62" lon="10"/>
|
||||||
|
<node id="52102" version="1" timestamp="2011-11-11T00:11:11Z" lat="62" lon="11"/>
|
||||||
|
<node id="52103" version="1" timestamp="2011-11-11T00:11:11Z" lat="64" lon="10"/>
|
||||||
|
<node id="52104" version="1" timestamp="2011-11-11T00:11:11Z" lat="64" lon="11"/>
|
||||||
|
<way id="52111" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
|
<nd ref="52101"/>
|
||||||
|
<nd ref="52102"/>
|
||||||
|
<nd ref="52103"/>
|
||||||
|
<nd ref="52104"/>
|
||||||
|
<nd ref="52101"/>
|
||||||
|
<tag k="building" v="yes"/>
|
||||||
|
</way>
|
||||||
|
<relation id="52121" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
|
<member type="way" ref="52111" role="outer"/>
|
||||||
|
<tag k="type" v="multipolygon"/>
|
||||||
|
</relation>
|
||||||
|
|
||||||
|
|
||||||
<!-- zig-zag line with coords internaly cached in differend deltacoords bunches -->
|
<!-- zig-zag line with coords internaly cached in differend deltacoords bunches -->
|
||||||
<node id="60001" version="1" timestamp="2011-11-11T00:11:11Z" lat="0.01" lon="20.00"/>
|
<node id="60001" version="1" timestamp="2011-11-11T00:11:11Z" lat="0.01" lon="20.00"/>
|
||||||
<node id="60002" version="1" timestamp="2011-11-11T00:11:11Z" lat="0.00" lon="20.01"/>
|
<node id="60002" version="1" timestamp="2011-11-11T00:11:11Z" lat="0.00" lon="20.01"/>
|
||||||
|
@ -1055,9 +1100,9 @@
|
||||||
<!-- test if additional create inserts duplicate elements (checks #66) -->
|
<!-- test if additional create inserts duplicate elements (checks #66) -->
|
||||||
<node id="201001" version="1" timestamp="2011-11-11T00:11:11Z" lat="32" lon="10"/>
|
<node id="201001" version="1" timestamp="2011-11-11T00:11:11Z" lat="32" lon="10"/>
|
||||||
<node id="201002" version="1" timestamp="2011-11-11T00:11:11Z" lat="32" lon="11"/>
|
<node id="201002" version="1" timestamp="2011-11-11T00:11:11Z" lat="32" lon="11"/>
|
||||||
<node id="201003" version="1" timestamp="2011-11-11T00:11:11Z" lat="34" lon="10"/>
|
<node id="201003" version="1" timestamp="2011-11-11T00:11:11Z" lat="34" lon="11"/>
|
||||||
<node id="201004" version="1" timestamp="2011-11-11T00:11:11Z" lat="34" lon="11"/>
|
<node id="201004" version="1" timestamp="2011-11-11T00:11:11Z" lat="34" lon="10"/>
|
||||||
<way id="201001" version="1" timestamp="2011-11-11T00:11:11Z">
|
<way id="201051" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
<nd ref="201001"/>
|
<nd ref="201001"/>
|
||||||
<nd ref="201002"/>
|
<nd ref="201002"/>
|
||||||
<nd ref="201003"/>
|
<nd ref="201003"/>
|
||||||
|
@ -1065,8 +1110,8 @@
|
||||||
<nd ref="201001"/>
|
<nd ref="201001"/>
|
||||||
<tag k="highway" v="residential"/>
|
<tag k="highway" v="residential"/>
|
||||||
</way>
|
</way>
|
||||||
<relation id="201001" version="1" timestamp="2011-11-11T00:11:11Z">
|
<relation id="201091" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
<member type="way" ref="201001" role="outer"/>
|
<member type="way" ref="201051" role="outer"/>
|
||||||
<tag k="type" v="multipolygon"/>
|
<tag k="type" v="multipolygon"/>
|
||||||
<tag k="landuse" v="park"/>
|
<tag k="landuse" v="park"/>
|
||||||
</relation>
|
</relation>
|
||||||
|
@ -1075,9 +1120,9 @@
|
||||||
relation (201101) with shared way (checks #65) -->
|
relation (201101) with shared way (checks #65) -->
|
||||||
<node id="201101" version="1" timestamp="2011-11-11T00:11:11Z" lat="32" lon="10"/>
|
<node id="201101" version="1" timestamp="2011-11-11T00:11:11Z" lat="32" lon="10"/>
|
||||||
<node id="201102" version="1" timestamp="2011-11-11T00:11:11Z" lat="32" lon="11"/>
|
<node id="201102" version="1" timestamp="2011-11-11T00:11:11Z" lat="32" lon="11"/>
|
||||||
<node id="201103" version="1" timestamp="2011-11-11T00:11:11Z" lat="34" lon="10"/>
|
<node id="201103" version="1" timestamp="2011-11-11T00:11:11Z" lat="34" lon="11"/>
|
||||||
<node id="201104" version="1" timestamp="2011-11-11T00:11:11Z" lat="34" lon="11"/>
|
<node id="201104" version="1" timestamp="2011-11-11T00:11:11Z" lat="34" lon="10"/>
|
||||||
<way id="201101" version="1" timestamp="2011-11-11T00:11:11Z">
|
<way id="201151" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
<nd ref="201101"/>
|
<nd ref="201101"/>
|
||||||
<nd ref="201102"/>
|
<nd ref="201102"/>
|
||||||
<nd ref="201103"/>
|
<nd ref="201103"/>
|
||||||
|
@ -1085,15 +1130,29 @@
|
||||||
<nd ref="201101"/>
|
<nd ref="201101"/>
|
||||||
<tag k="highway" v="residential"/>
|
<tag k="highway" v="residential"/>
|
||||||
</way>
|
</way>
|
||||||
<relation id="201101" version="1" timestamp="2011-11-11T00:11:11Z">
|
<relation id="201191" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
<member type="way" ref="201101" role="outer"/>
|
<member type="way" ref="201151" role="outer"/>
|
||||||
<tag k="type" v="multipolygon"/>
|
<tag k="type" v="multipolygon"/>
|
||||||
<tag k="landuse" v="park"/>
|
<tag k="landuse" v="park"/>
|
||||||
</relation>
|
</relation>
|
||||||
<relation id="201102" version="1" timestamp="2011-11-11T00:11:11Z">
|
<relation id="201192" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
<member type="way" ref="201101" role="outer"/>
|
<member type="way" ref="201151" role="outer"/>
|
||||||
<tag k="type" v="multipolygon"/>
|
<tag k="type" v="multipolygon"/>
|
||||||
<tag k="landuse" v="forest"/>
|
<tag k="landuse" v="forest"/>
|
||||||
</relation>
|
</relation>
|
||||||
|
|
||||||
|
<!-- test that relations with unsupported types are not inserted with updates -->
|
||||||
|
<node id="201201" version="1" timestamp="2011-11-11T00:11:11Z" lat="32" lon="20"/>
|
||||||
|
<node id="201202" version="1" timestamp="2011-11-11T00:11:11Z" lat="32" lon="21"/>
|
||||||
|
<node id="201203" version="1" timestamp="2011-11-11T00:11:11Z" lat="34" lon="21"/>
|
||||||
|
<node id="201204" version="1" timestamp="2011-11-11T00:11:11Z" lat="34" lon="20"/>
|
||||||
|
<way id="201251" version="1" timestamp="2011-11-11T00:11:11Z">
|
||||||
|
<nd ref="201201"/>
|
||||||
|
<nd ref="201202"/>
|
||||||
|
<nd ref="201203"/>
|
||||||
|
<nd ref="201204"/>
|
||||||
|
<nd ref="201201"/>
|
||||||
|
<tag k="landuse" v="park"/>
|
||||||
|
</way>
|
||||||
|
|
||||||
</osm>
|
</osm>
|
||||||
|
|
|
@ -1,525 +0,0 @@
|
||||||
import unittest
|
|
||||||
|
|
||||||
import helper as t
|
|
||||||
|
|
||||||
mapping_file = 'complete_db_mapping.json'
|
|
||||||
|
|
||||||
def setup():
|
|
||||||
t.setup()
|
|
||||||
|
|
||||||
def teardown():
|
|
||||||
t.teardown()
|
|
||||||
|
|
||||||
#######################################################################
|
|
||||||
def test_import():
|
|
||||||
"""Import succeeds"""
|
|
||||||
t.drop_schemas()
|
|
||||||
assert not t.table_exists('osm_roads', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
t.imposm3_import(t.db_conf, './build/complete_db.pbf', mapping_file)
|
|
||||||
assert t.table_exists('osm_roads', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
|
|
||||||
def test_deploy():
|
|
||||||
"""Deploy succeeds"""
|
|
||||||
assert not t.table_exists('osm_roads', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
t.imposm3_deploy(t.db_conf, mapping_file)
|
|
||||||
assert t.table_exists('osm_roads', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
assert not t.table_exists('osm_roads', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
|
|
||||||
#######################################################################
|
|
||||||
|
|
||||||
def test_imported_landusage():
|
|
||||||
"""Multipolygon relation is inserted"""
|
|
||||||
t.assert_cached_node(1001, (13, 47.5))
|
|
||||||
landusage_1001 = t.query_row(t.db_conf, 'osm_landusages', -1001)
|
|
||||||
# point in polygon
|
|
||||||
assert landusage_1001['geometry'].intersects(t.merc_point(13.4, 47.5))
|
|
||||||
# hole in multipolygon relation
|
|
||||||
assert not landusage_1001['geometry'].intersects(t.merc_point(14.75, 47.75))
|
|
||||||
|
|
||||||
def test_missing_nodes():
|
|
||||||
"""Cache does not contain nodes from previous imports"""
|
|
||||||
t.assert_missing_node(10001)
|
|
||||||
t.assert_missing_node(10002)
|
|
||||||
place_10000 = t.query_row(t.db_conf, 'osm_places', 10000)
|
|
||||||
assert place_10000['name'] == 'Foo', place_10000
|
|
||||||
|
|
||||||
def test_name_tags():
|
|
||||||
"""Road contains multiple names"""
|
|
||||||
road = t.query_row(t.db_conf, 'osm_roads', 1101)
|
|
||||||
assert road['name'] == 'name', road
|
|
||||||
assert road['name:de'] == 'name:de', road
|
|
||||||
assert road['name_en'] == 'name:en', road
|
|
||||||
|
|
||||||
def test_landusage_to_waterarea_1():
|
|
||||||
"""Parks inserted into landusages"""
|
|
||||||
t.assert_cached_way(11001)
|
|
||||||
t.assert_cached_way(12001)
|
|
||||||
t.assert_cached_way(13001)
|
|
||||||
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_waterareas', 11001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_waterareas', -12001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_waterareas', -13001)
|
|
||||||
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_waterareas_gen0', 11001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_waterareas_gen0', -12001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_waterareas_gen0', -13001)
|
|
||||||
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_waterareas_gen1', 11001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_waterareas_gen1', -12001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_waterareas_gen1', -13001)
|
|
||||||
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', 11001)['type'] == 'park'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', -12001)['type'] == 'park'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', -13001)['type'] == 'park'
|
|
||||||
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages_gen0', 11001)['type'] == 'park'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages_gen0', -12001)['type'] == 'park'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages_gen0', -13001)['type'] == 'park'
|
|
||||||
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages_gen1', 11001)['type'] == 'park'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages_gen1', -12001)['type'] == 'park'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages_gen1', -13001)['type'] == 'park'
|
|
||||||
|
|
||||||
|
|
||||||
def test_changed_hole_tags_1():
|
|
||||||
"""Multipolygon relation with untagged hole"""
|
|
||||||
t.assert_cached_way(14001)
|
|
||||||
t.assert_cached_way(14011)
|
|
||||||
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_waterareas', 14011)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_waterareas', -14011)
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', -14001)['type'] == 'park'
|
|
||||||
|
|
||||||
def test_split_outer_multipolygon_way_1():
|
|
||||||
"""Single outer way of multipolygon was inserted."""
|
|
||||||
park_15001 = t.query_row(t.db_conf, 'osm_landusages', -15001)
|
|
||||||
assert park_15001['type'] == 'park'
|
|
||||||
t.assert_almost_equal(park_15001['geometry'].area, 9816216452, -1)
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 15002) == None
|
|
||||||
|
|
||||||
def test_merge_outer_multipolygon_way_1():
|
|
||||||
"""Splitted outer way of multipolygon was inserted."""
|
|
||||||
park_16001 = t.query_row(t.db_conf, 'osm_landusages', -16001)
|
|
||||||
assert park_16001['type'] == 'park'
|
|
||||||
t.assert_almost_equal(park_16001['geometry'].area, 12779350582, -1)
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 16002)['type'] == 'residential'
|
|
||||||
|
|
||||||
def test_broken_multipolygon_ways():
|
|
||||||
"""MultiPolygons with broken outer ways are handled."""
|
|
||||||
# outer way does not merge (17002 has one node)
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', -17001) == None
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 17001)['type'] == 'residential'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 17002) == None
|
|
||||||
|
|
||||||
# outer way does not merge (17102 has no nodes)
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', -17101) == None
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 17101)['type'] == 'residential'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 17102) == None
|
|
||||||
|
|
||||||
def test_node_way_inserted_twice():
|
|
||||||
"""Way with multiple mappings is inserted twice in same table"""
|
|
||||||
rows = t.query_row(t.db_conf, 'osm_roads', 18001)
|
|
||||||
rows.sort(key=lambda x: x['type'])
|
|
||||||
|
|
||||||
assert rows[0]['type'] == 'residential'
|
|
||||||
assert rows[1]['type'] == 'tram'
|
|
||||||
|
|
||||||
def test_outer_way_not_inserted():
|
|
||||||
"""Outer way with different tag is not inserted twice into same table"""
|
|
||||||
farm = t.query_row(t.db_conf, 'osm_landusages', -19001)
|
|
||||||
assert farm['type'] == 'farmland'
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages', 19001)
|
|
||||||
|
|
||||||
farmyard = t.query_row(t.db_conf, 'osm_landusages', 19002)
|
|
||||||
assert farmyard['type'] == 'farmyard'
|
|
||||||
|
|
||||||
def test_outer_way_inserted():
|
|
||||||
"""Outer way with different tag is inserted twice into different table"""
|
|
||||||
farm = t.query_row(t.db_conf, 'osm_landusages', 19101)
|
|
||||||
assert farm['type'] == 'farm'
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages', -19101)
|
|
||||||
|
|
||||||
farmyard = t.query_row(t.db_conf, 'osm_landusages', 19102)
|
|
||||||
assert farmyard['type'] == 'farmyard'
|
|
||||||
|
|
||||||
admin = t.query_row(t.db_conf, 'osm_admin', -19101)
|
|
||||||
assert admin['type'] == 'administrative'
|
|
||||||
|
|
||||||
def test_node_way_ref_after_delete_1():
|
|
||||||
"""Nodes refereces way"""
|
|
||||||
data = t.cache_query(nodes=[20001, 20002], deps=True)
|
|
||||||
assert '20001' in data['nodes']['20001']['ways']
|
|
||||||
assert '20001' in data['nodes']['20002']['ways']
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 20001)['type'] == 'residential'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_barrierpoints', 20001)['type'] == 'block'
|
|
||||||
|
|
||||||
def test_way_rel_ref_after_delete_1():
|
|
||||||
"""Ways references relation"""
|
|
||||||
data = t.cache_query(ways=[21001], deps=True)
|
|
||||||
assert data['ways']['21001']['relations'].keys() == ['21001']
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 21001)['type'] == 'residential'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', -21001)['type'] == 'park'
|
|
||||||
|
|
||||||
def test_relation_way_not_inserted():
|
|
||||||
"""Part of relation was inserted only once."""
|
|
||||||
park = t.query_row(t.db_conf, 'osm_landusages', -9001)
|
|
||||||
assert park['type'] == 'park'
|
|
||||||
assert park['name'] == 'rel 9001'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', 9009) == None
|
|
||||||
|
|
||||||
park = t.query_row(t.db_conf, 'osm_landusages', -9101)
|
|
||||||
assert park['type'] == 'park'
|
|
||||||
assert park['name'] == 'rel 9101'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', 9109) == None
|
|
||||||
|
|
||||||
scrub = t.query_row(t.db_conf, 'osm_landusages', 9110)
|
|
||||||
assert scrub['type'] == 'scrub'
|
|
||||||
|
|
||||||
def test_relation_ways_inserted():
|
|
||||||
"""Outer ways of multipolygon are inserted. """
|
|
||||||
park = t.query_row(t.db_conf, 'osm_landusages', -9201)
|
|
||||||
assert park['type'] == 'park'
|
|
||||||
assert park['name'] == '9209'
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages', 9201)
|
|
||||||
|
|
||||||
# outer ways of multipolygon stand for their own
|
|
||||||
road = t.query_row(t.db_conf, 'osm_roads', 9209)
|
|
||||||
assert road['type'] == 'secondary'
|
|
||||||
assert road['name'] == '9209'
|
|
||||||
road = t.query_row(t.db_conf, 'osm_roads', 9210)
|
|
||||||
assert road['type'] == 'residential'
|
|
||||||
assert road['name'] == '9210'
|
|
||||||
|
|
||||||
park = t.query_row(t.db_conf, 'osm_landusages', -9301)
|
|
||||||
assert park['type'] == 'park'
|
|
||||||
assert park['name'] == '' # no name on relation
|
|
||||||
|
|
||||||
# outer ways of multipolygon stand for their own
|
|
||||||
road = t.query_row(t.db_conf, 'osm_roads', 9309)
|
|
||||||
assert road['type'] == 'secondary'
|
|
||||||
assert road['name'] == '9309'
|
|
||||||
road = t.query_row(t.db_conf, 'osm_roads', 9310)
|
|
||||||
assert road['type'] == 'residential'
|
|
||||||
assert road['name'] == '9310'
|
|
||||||
|
|
||||||
def test_relation_way_inserted():
|
|
||||||
"""Part of relation was inserted twice."""
|
|
||||||
park = t.query_row(t.db_conf, 'osm_landusages', -8001)
|
|
||||||
assert park['type'] == 'park'
|
|
||||||
assert park['name'] == 'rel 8001'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 8009)["type"] == 'residential'
|
|
||||||
|
|
||||||
def test_single_node_ways_not_inserted():
|
|
||||||
"""Ways with single/duplicate nodes are not inserted."""
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_roads', 30001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_roads', 30002)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_roads', 30003)
|
|
||||||
|
|
||||||
def test_polygon_with_duplicate_nodes_is_valid():
|
|
||||||
"""Polygon with duplicate nodes is valid."""
|
|
||||||
geom = t.query_row(t.db_conf, 'osm_landusages', 30005)['geometry']
|
|
||||||
assert geom.is_valid
|
|
||||||
assert len(geom.exterior.coords) == 4
|
|
||||||
|
|
||||||
def test_incomplete_polygons():
|
|
||||||
"""Non-closed/incomplete polygons are not inserted."""
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages', 30004)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages', 30006)
|
|
||||||
|
|
||||||
def test_residential_to_secondary():
|
|
||||||
"""Residential road is not in roads_gen0/1."""
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 40001)['type'] == 'residential'
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_roads_gen0', 40001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_roads_gen1', 40001)
|
|
||||||
|
|
||||||
def test_relation_before_remove():
|
|
||||||
"""Relation and way is inserted."""
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', 50011)['type'] == 'yes'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', -50021)['type'] == 'park'
|
|
||||||
|
|
||||||
def test_relation_without_tags():
|
|
||||||
"""Relation without tags is inserted."""
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', 50111) == None
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', -50121)['type'] == 'yes'
|
|
||||||
|
|
||||||
def test_duplicate_ids():
|
|
||||||
"""Relation/way with same ID is inserted."""
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', 51001)['type'] == 'way'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', -51001)['type'] == 'mp'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', 51011)['type'] == 'way'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', -51011)['type'] == 'mp'
|
|
||||||
|
|
||||||
def test_generalized_banana_polygon_is_valid():
|
|
||||||
"""Generalized polygons are valid."""
|
|
||||||
park = t.query_row(t.db_conf, 'osm_landusages', 7101)
|
|
||||||
# geometry is not valid
|
|
||||||
assert not park['geometry'].is_valid, park
|
|
||||||
park = t.query_row(t.db_conf, 'osm_landusages_gen0', 7101)
|
|
||||||
# but simplified geometies are valid
|
|
||||||
assert park['geometry'].is_valid, park
|
|
||||||
park = t.query_row(t.db_conf, 'osm_landusages_gen1', 7101)
|
|
||||||
assert park['geometry'].is_valid, park
|
|
||||||
|
|
||||||
def test_generalized_linestring_is_valid():
|
|
||||||
"""Generalized linestring is valid."""
|
|
||||||
road = t.query_row(t.db_conf, 'osm_roads', 7201)
|
|
||||||
# geometry is not simple, but valid
|
|
||||||
# check that geometry 'survives' simplification
|
|
||||||
assert not road['geometry'].is_simple, road['geometry'].wkt
|
|
||||||
assert road['geometry'].is_valid, road['geometry'].wkt
|
|
||||||
assert road['geometry'].length > 1000000
|
|
||||||
road = t.query_row(t.db_conf, 'osm_roads_gen0', 7201)
|
|
||||||
# but simplified geometies are simple
|
|
||||||
assert road['geometry'].is_valid, road['geometry'].wkt
|
|
||||||
assert road['geometry'].length > 1000000
|
|
||||||
road = t.query_row(t.db_conf, 'osm_roads_gen1', 7201)
|
|
||||||
assert road['geometry'].is_valid, road['geometry'].wkt
|
|
||||||
assert road['geometry'].length > 1000000
|
|
||||||
|
|
||||||
def test_ring_with_gap():
|
|
||||||
"""Multipolygon and way with gap (overlapping but different endpoints) gets closed"""
|
|
||||||
park = t.query_row(t.db_conf, 'osm_landusages', -7301)
|
|
||||||
assert park['geometry'].is_valid, park
|
|
||||||
|
|
||||||
park = t.query_row(t.db_conf, 'osm_landusages', 7311)
|
|
||||||
assert park['geometry'].is_valid, park
|
|
||||||
|
|
||||||
def test_updated_nodes1():
|
|
||||||
"""Zig-Zag line is inserted."""
|
|
||||||
road = t.query_row(t.db_conf, 'osm_roads', 60000)
|
|
||||||
t.assert_almost_equal(road['geometry'].length, 14035.61150207768)
|
|
||||||
|
|
||||||
def test_update_node_to_coord_1():
|
|
||||||
"""Node is inserted with tag."""
|
|
||||||
coords = t.cache_query(nodes=(70001, 70002))
|
|
||||||
assert coords['nodes']["70001"]["tags"] == {"amenity": "police"}
|
|
||||||
assert "tags" not in coords['nodes']["70002"]
|
|
||||||
|
|
||||||
assert t.query_row(t.db_conf, 'osm_amenities', 70001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_amenities', 70002)
|
|
||||||
|
|
||||||
def test_enumerate_key():
|
|
||||||
"""Enumerate from key."""
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', 100001)['enum'] == 1
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', 100002)['enum'] == 0
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', 100003)['enum'] == 15
|
|
||||||
|
|
||||||
|
|
||||||
#######################################################################
|
|
||||||
def test_update():
|
|
||||||
"""Diff import applies"""
|
|
||||||
t.imposm3_update(t.db_conf, './build/complete_db.osc.gz', mapping_file)
|
|
||||||
#######################################################################
|
|
||||||
|
|
||||||
|
|
||||||
def test_no_duplicates():
|
|
||||||
"""
|
|
||||||
Relations/ways are only inserted once
|
|
||||||
Checks #66
|
|
||||||
"""
|
|
||||||
highways = t.query_duplicates(t.db_conf, 'osm_roads')
|
|
||||||
# one duplicate for test_node_way_inserted_twice is expected
|
|
||||||
assert highways == [[18001, 2]]
|
|
||||||
landusages = t.query_duplicates(t.db_conf, 'osm_landusages')
|
|
||||||
assert not landusages
|
|
||||||
|
|
||||||
def test_updated_landusage():
|
|
||||||
"""Multipolygon relation was modified"""
|
|
||||||
t.assert_cached_node(1001, (13.5, 47.5))
|
|
||||||
landusage_1001 = t.query_row(t.db_conf, 'osm_landusages', -1001)
|
|
||||||
# point not in polygon after update
|
|
||||||
assert not landusage_1001['geometry'].intersects(t.merc_point(13.4, 47.5))
|
|
||||||
|
|
||||||
def test_partial_delete():
|
|
||||||
"""Deleted relation but nodes are still cached"""
|
|
||||||
t.assert_cached_node(2001)
|
|
||||||
t.assert_cached_way(2001)
|
|
||||||
t.assert_cached_way(2002)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages', -2001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages', 2001)
|
|
||||||
|
|
||||||
def test_updated_nodes():
|
|
||||||
"""Nodes were added, modified or deleted"""
|
|
||||||
t.assert_missing_node(10000)
|
|
||||||
t.assert_cached_node(10001, (10.0, 40.0))
|
|
||||||
t.assert_cached_node(10002, (10.1, 40.0))
|
|
||||||
place_10001 = t.query_row(t.db_conf, 'osm_places', 10001)
|
|
||||||
assert place_10001['name'] == 'Bar', place_10001
|
|
||||||
place_10002 = t.query_row(t.db_conf, 'osm_places', 10002)
|
|
||||||
assert place_10002['name'] == 'Baz', place_10002
|
|
||||||
|
|
||||||
def test_landusage_to_waterarea_2():
|
|
||||||
"""Parks converted to water moved from landusages to waterareas"""
|
|
||||||
t.assert_cached_way(11001)
|
|
||||||
t.assert_cached_way(12001)
|
|
||||||
t.assert_cached_way(13001)
|
|
||||||
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages', 11001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages', -12001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages', -13001)
|
|
||||||
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages_gen0', 11001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages_gen0', -12001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages_gen0', -13001)
|
|
||||||
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages_gen1', 11001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages_gen1', -12001)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_landusages_gen1', -13001)
|
|
||||||
|
|
||||||
assert t.query_row(t.db_conf, 'osm_waterareas', 11001)['type'] == 'water'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_waterareas', -12001)['type'] == 'water'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_waterareas', -13001)['type'] == 'water'
|
|
||||||
|
|
||||||
assert t.query_row(t.db_conf, 'osm_waterareas_gen0', 11001)['type'] == 'water'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_waterareas_gen0', -12001)['type'] == 'water'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_waterareas_gen0', -13001)['type'] == 'water'
|
|
||||||
|
|
||||||
assert t.query_row(t.db_conf, 'osm_waterareas_gen1', 11001)['type'] == 'water'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_waterareas_gen1', -12001)['type'] == 'water'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_waterareas_gen1', -13001)['type'] == 'water'
|
|
||||||
|
|
||||||
def test_changed_hole_tags_2():
|
|
||||||
"""Newly tagged hole is inserted"""
|
|
||||||
t.assert_cached_way(14001)
|
|
||||||
t.assert_cached_way(14011)
|
|
||||||
|
|
||||||
assert t.query_row(t.db_conf, 'osm_waterareas', 14011)['type'] == 'water'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', -14001)['type'] == 'park'
|
|
||||||
t.assert_almost_equal(t.query_row(t.db_conf, 'osm_waterareas', 14011)['geometry'].area, 26672000000, -6)
|
|
||||||
t.assert_almost_equal(t.query_row(t.db_conf, 'osm_landusages', -14001)['geometry'].area, 10373600000, -6)
|
|
||||||
|
|
||||||
def test_split_outer_multipolygon_way_2():
|
|
||||||
"""Splitted outer way of multipolygon was inserted"""
|
|
||||||
data = t.cache_query(ways=[15001, 15002], deps=True)
|
|
||||||
assert data['ways']['15001']['relations'].keys() == ['15001']
|
|
||||||
assert data['ways']['15002']['relations'].keys() == ['15001']
|
|
||||||
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', 15001) == None
|
|
||||||
park_15001 = t.query_row(t.db_conf, 'osm_landusages', -15001)
|
|
||||||
assert park_15001['type'] == 'park'
|
|
||||||
t.assert_almost_equal(park_15001['geometry'].area, 9816216452, -1)
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 15002)['type'] == 'residential'
|
|
||||||
|
|
||||||
def test_merge_outer_multipolygon_way_2():
|
|
||||||
"""Merged outer way of multipolygon was inserted"""
|
|
||||||
data = t.cache_query(ways=[16001, 16002], deps=True)
|
|
||||||
assert data['ways']['16001']['relations'].keys() == ['16001']
|
|
||||||
assert data['ways']['16002'] == None
|
|
||||||
|
|
||||||
data = t.cache_query(relations=[16001], full=True)
|
|
||||||
assert sorted(data['relations']['16001']['ways'].keys()) == ['16001', '16011']
|
|
||||||
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', 16001) == None
|
|
||||||
park_16001 = t.query_row(t.db_conf, 'osm_landusages', -16001)
|
|
||||||
assert park_16001['type'] == 'park'
|
|
||||||
t.assert_almost_equal(park_16001['geometry'].area, 12779350582, -1)
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 16002) == None
|
|
||||||
|
|
||||||
def test_node_way_ref_after_delete_2():
|
|
||||||
"""Node does not referece deleted way"""
|
|
||||||
data = t.cache_query(nodes=[20001, 20002], deps=True)
|
|
||||||
assert 'ways' not in data['nodes']['20001']
|
|
||||||
assert data['nodes']['20002'] == None
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 20001) == None
|
|
||||||
assert t.query_row(t.db_conf, 'osm_barrierpoints', 20001)['type'] == 'block'
|
|
||||||
|
|
||||||
def test_way_rel_ref_after_delete_2():
|
|
||||||
"""Way does not referece deleted relation"""
|
|
||||||
data = t.cache_query(ways=[21001], deps=True)
|
|
||||||
assert 'relations' not in data['ways']['21001']
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 21001)['type'] == 'residential'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', 21001) == None
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', -21001) == None
|
|
||||||
|
|
||||||
def test_residential_to_secondary2():
|
|
||||||
"""New secondary (from residential) is now in roads_gen0/1."""
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 40001)['type'] == 'secondary'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads_gen0', 40001)['type'] == 'secondary'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads_gen1', 40001)['type'] == 'secondary'
|
|
||||||
|
|
||||||
def test_relation_after_remove():
|
|
||||||
"""Relation is deleted and way is still present."""
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', 50011)['type'] == 'yes'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', 50021) == None
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', -50021) == None
|
|
||||||
|
|
||||||
def test_relation_without_tags2():
|
|
||||||
"""Relation without tags is removed."""
|
|
||||||
t.cache_query(ways=[50111], deps=True)
|
|
||||||
assert t.cache_query(relations=[50121], deps=True)['relations']["50121"] == None
|
|
||||||
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', 50111)['type'] == 'yes'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', 50121) == None
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', -50121) == None
|
|
||||||
|
|
||||||
def test_duplicate_ids2():
|
|
||||||
"""Only relation/way with same ID was deleted."""
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', 51001)['type'] == 'way'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', -51001) == None
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', -51011)['type'] == 'mp'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_buildings', 51011) == None
|
|
||||||
|
|
||||||
def test_updated_way2():
|
|
||||||
"""All nodes of straightened way are updated."""
|
|
||||||
road = t.query_row(t.db_conf, 'osm_roads', 60000)
|
|
||||||
# new length 0.1 degree
|
|
||||||
t.assert_almost_equal(road['geometry'].length, 20037508.342789244/180.0/10.0)
|
|
||||||
|
|
||||||
def test_update_node_to_coord_2():
|
|
||||||
"""Node is becomes coord after tags are removed."""
|
|
||||||
coords = t.cache_query(nodes=(70001, 70002))
|
|
||||||
|
|
||||||
assert "tags" not in coords['nodes']["70001"]
|
|
||||||
assert coords['nodes']["70002"]["tags"] == {"amenity": "police"}
|
|
||||||
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_amenities', 70001)
|
|
||||||
assert t.query_row(t.db_conf, 'osm_amenities', 70002)
|
|
||||||
|
|
||||||
def test_no_duplicate_insert():
|
|
||||||
"""
|
|
||||||
Relation is not inserted again if a nother relation with the same way was modified
|
|
||||||
Checks #65
|
|
||||||
"""
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', -201101)['type'] == 'park'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_landusages', -201102)['type'] == 'forest'
|
|
||||||
assert t.query_row(t.db_conf, 'osm_roads', 201101)['type'] == 'residential'
|
|
||||||
|
|
||||||
#######################################################################
|
|
||||||
def test_deploy_and_revert_deploy():
|
|
||||||
"""Revert deploy succeeds"""
|
|
||||||
assert not t.table_exists('osm_roads', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
assert t.table_exists('osm_roads', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
assert not t.table_exists('osm_roads', schema=t.TEST_SCHEMA_BACKUP)
|
|
||||||
|
|
||||||
# import again to have a new import schema
|
|
||||||
t.imposm3_import(t.db_conf, './build/complete_db.pbf', mapping_file)
|
|
||||||
assert t.table_exists('osm_roads', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
|
|
||||||
t.imposm3_deploy(t.db_conf, mapping_file)
|
|
||||||
assert not t.table_exists('osm_roads', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
assert t.table_exists('osm_roads', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
assert t.table_exists('osm_roads', schema=t.TEST_SCHEMA_BACKUP)
|
|
||||||
|
|
||||||
t.imposm3_revert_deploy(t.db_conf, mapping_file)
|
|
||||||
assert t.table_exists('osm_roads', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
assert t.table_exists('osm_roads', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
assert not t.table_exists('osm_roads', schema=t.TEST_SCHEMA_BACKUP)
|
|
||||||
|
|
||||||
def test_remove_backup():
|
|
||||||
"""Remove backup succeeds"""
|
|
||||||
assert t.table_exists('osm_roads', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
assert t.table_exists('osm_roads', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
assert not t.table_exists('osm_roads', schema=t.TEST_SCHEMA_BACKUP)
|
|
||||||
|
|
||||||
t.imposm3_deploy(t.db_conf, mapping_file)
|
|
||||||
|
|
||||||
assert not t.table_exists('osm_roads', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
assert t.table_exists('osm_roads', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
assert t.table_exists('osm_roads', schema=t.TEST_SCHEMA_BACKUP)
|
|
||||||
|
|
||||||
t.imposm3_remove_backups(t.db_conf, mapping_file)
|
|
||||||
|
|
||||||
assert not t.table_exists('osm_roads', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
assert t.table_exists('osm_roads', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
assert not t.table_exists('osm_roads', schema=t.TEST_SCHEMA_BACKUP)
|
|
||||||
|
|
|
@ -0,0 +1,740 @@
|
||||||
|
package test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"database/sql"
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
"github.com/omniscale/imposm3/cache"
|
||||||
|
|
||||||
|
"github.com/omniscale/imposm3/element"
|
||||||
|
"github.com/omniscale/imposm3/geom"
|
||||||
|
"github.com/omniscale/imposm3/proj"
|
||||||
|
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/omniscale/imposm3/geom/geos"
|
||||||
|
)
|
||||||
|
|
||||||
|
var ts importTestSuite
|
||||||
|
|
||||||
|
func TestPrepare(t *testing.T) {
|
||||||
|
ts.dir = "/tmp/imposm3test"
|
||||||
|
ts.config = importConfig{
|
||||||
|
connection: "postgis://",
|
||||||
|
cacheDir: ts.dir,
|
||||||
|
osmFileName: "build/complete_db.pbf",
|
||||||
|
mappingFileName: "complete_db_mapping.json",
|
||||||
|
}
|
||||||
|
ts.g = geos.NewGeos()
|
||||||
|
|
||||||
|
var err error
|
||||||
|
ts.db, err = sql.Open("postgres", "sslmode=disable")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ts.dropSchemas()
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestImport(t *testing.T) {
|
||||||
|
if ts.tableExists(t, dbschemaImport, "osm_roads") != false {
|
||||||
|
t.Fatalf("table osm_roads exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
ts.importOsm(t)
|
||||||
|
if ts.tableExists(t, dbschemaImport, "osm_roads") != true {
|
||||||
|
t.Fatalf("table osm_roads does not exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeploy(t *testing.T) {
|
||||||
|
ts.deployOsm(t)
|
||||||
|
if ts.tableExists(t, dbschemaImport, "osm_roads") != false {
|
||||||
|
t.Fatalf("table osm_roads exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
if ts.tableExists(t, dbschemaProduction, "osm_roads") != true {
|
||||||
|
t.Fatalf("table osm_roads does not exists in schema %s", dbschemaProduction)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLandusageToWaterarea1(t *testing.T) {
|
||||||
|
// Parks inserted into landusages
|
||||||
|
cache := ts.cache(t)
|
||||||
|
defer cache.Close()
|
||||||
|
assertCachedWay(t, cache, 11001)
|
||||||
|
assertCachedWay(t, cache, 12001)
|
||||||
|
assertCachedWay(t, cache, 13001)
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_waterareas", 11001, Missing, nil},
|
||||||
|
{"osm_waterareas", -12001, Missing, nil},
|
||||||
|
{"osm_waterareas", -13001, Missing, nil},
|
||||||
|
|
||||||
|
{"osm_waterareas_gen0", 11001, Missing, nil},
|
||||||
|
{"osm_waterareas_gen0", -12001, Missing, nil},
|
||||||
|
{"osm_waterareas_gen0", -13001, Missing, nil},
|
||||||
|
|
||||||
|
{"osm_waterareas_gen1", 11001, Missing, nil},
|
||||||
|
{"osm_waterareas_gen1", -12001, Missing, nil},
|
||||||
|
{"osm_waterareas_gen1", -13001, Missing, nil},
|
||||||
|
|
||||||
|
{"osm_landusages", 11001, "park", nil},
|
||||||
|
{"osm_landusages", -12001, "park", nil},
|
||||||
|
{"osm_landusages", -13001, "park", nil},
|
||||||
|
|
||||||
|
{"osm_landusages_gen0", 11001, "park", nil},
|
||||||
|
{"osm_landusages_gen0", -12001, "park", nil},
|
||||||
|
{"osm_landusages_gen0", -13001, "park", nil},
|
||||||
|
|
||||||
|
{"osm_landusages_gen1", 11001, "park", nil},
|
||||||
|
{"osm_landusages_gen1", -12001, "park", nil},
|
||||||
|
{"osm_landusages_gen1", -13001, "park", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestChangedHoleTags1(t *testing.T) {
|
||||||
|
// Multipolygon relation with untagged hole
|
||||||
|
cache := ts.cache(t)
|
||||||
|
defer cache.Close()
|
||||||
|
assertCachedWay(t, cache, 14001)
|
||||||
|
assertCachedWay(t, cache, 14011)
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_waterareas", 14011, Missing, nil},
|
||||||
|
{"osm_waterareas", -14011, Missing, nil},
|
||||||
|
{"osm_landusages", -14001, "park", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSplitOuterMultipolygonWay1(t *testing.T) {
|
||||||
|
// Single outer way of multipolygon was inserted.
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_roads", 15002, Missing, nil},
|
||||||
|
{"osm_landusages", -15001, "park", nil},
|
||||||
|
})
|
||||||
|
assertArea(t, checkElem{"osm_landusages", -15001, "park", nil}, 9816216452)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMergeOuterMultipolygonWay1(t *testing.T) {
|
||||||
|
// Splitted outer way of multipolygon was inserted.
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", -16001, "park", nil},
|
||||||
|
{"osm_roads", 16002, "residential", nil},
|
||||||
|
})
|
||||||
|
assertArea(t, checkElem{"osm_landusages", -16001, "park", nil}, 12779350582)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBrokenMultipolygonWays(t *testing.T) {
|
||||||
|
// MultiPolygons with broken outer ways are handled.
|
||||||
|
// outer way does not merge (17002 has one node)
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", -17001, Missing, nil},
|
||||||
|
{"osm_roads", 17001, "residential", nil},
|
||||||
|
{"osm_roads", 17002, Missing, nil},
|
||||||
|
})
|
||||||
|
|
||||||
|
// outer way does not merge (17102 has no nodes)
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", -17101, Missing, nil},
|
||||||
|
{"osm_roads", 17101, "residential", nil},
|
||||||
|
{"osm_roads", 17102, Missing, nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNodeWayInsertedTwice(t *testing.T) {
|
||||||
|
// Way with multiple mappings is inserted twice in same table
|
||||||
|
rows := ts.queryRows(t, "osm_roads", 18001)
|
||||||
|
if len(rows) != 2 || rows[0].osmType != "residential" || rows[1].osmType != "tram" {
|
||||||
|
t.Errorf("unexpected roads: %v", rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOuterWayNotInserted(t *testing.T) {
|
||||||
|
// Outer way with different tag is not inserted twice into same table
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", -19001, "farmland", nil},
|
||||||
|
{"osm_landusages", 19002, "farmyard", nil},
|
||||||
|
{"osm_landusages", 19001, Missing, nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOuterWayInserted(t *testing.T) {
|
||||||
|
// Outer way with different tag is inserted twice into different table
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", 19101, "farm", nil},
|
||||||
|
{"osm_landusages", 19102, "farmyard", nil},
|
||||||
|
{"osm_admin", -19101, "administrative", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNodeWayRefAfterDelete1(t *testing.T) {
|
||||||
|
// Nodes references way
|
||||||
|
|
||||||
|
cache := ts.diffCache(t)
|
||||||
|
defer cache.Close()
|
||||||
|
if ids := cache.Coords.Get(20001); len(ids) != 1 || ids[0] != 20001 {
|
||||||
|
t.Error("node does not references way")
|
||||||
|
}
|
||||||
|
if ids := cache.Coords.Get(20002); len(ids) != 1 || ids[0] != 20001 {
|
||||||
|
t.Error("node does not references way")
|
||||||
|
}
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_roads", 20001, "residential", nil},
|
||||||
|
{"osm_barrierpoints", 20001, "block", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWayRelRefAfterDelete1(t *testing.T) {
|
||||||
|
// Ways references relation
|
||||||
|
|
||||||
|
cache := ts.diffCache(t)
|
||||||
|
defer cache.Close()
|
||||||
|
if ids := cache.Ways.Get(21001); len(ids) != 1 || ids[0] != 21001 {
|
||||||
|
t.Error("way does not references relation")
|
||||||
|
}
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_roads", 21001, "residential", nil},
|
||||||
|
{"osm_landusages", -21001, "park", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelationWayNotInserted(t *testing.T) {
|
||||||
|
// Part of relation was inserted only once.
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", -9001, "park", map[string]string{"name": "rel 9001"}},
|
||||||
|
{"osm_landusages", 9009, Missing, nil},
|
||||||
|
{"osm_landusages", -9101, "park", map[string]string{"name": "rel 9101"}},
|
||||||
|
{"osm_landusages", 9109, Missing, nil},
|
||||||
|
{"osm_landusages", 9110, "scrub", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelationWaysInserted(t *testing.T) {
|
||||||
|
// Outer ways of multipolygon are inserted.
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", -9201, "park", map[string]string{"name": "9209"}},
|
||||||
|
{"osm_landusages", 9201, Missing, nil},
|
||||||
|
// outer ways of multipolygon stand for their own
|
||||||
|
{"osm_roads", 9209, "secondary", map[string]string{"name": "9209"}},
|
||||||
|
{"osm_roads", 9210, "residential", map[string]string{"name": "9210"}},
|
||||||
|
|
||||||
|
// no name on relation
|
||||||
|
{"osm_landusages", -9301, "park", map[string]string{"name": ""}},
|
||||||
|
// outer ways of multipolygon stand for their own
|
||||||
|
{"osm_roads", 9309, "secondary", map[string]string{"name": "9309"}},
|
||||||
|
{"osm_roads", 9310, "residential", map[string]string{"name": "9310"}},
|
||||||
|
})
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelationWayInserted(t *testing.T) {
|
||||||
|
// Part of relation was inserted twice.
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", -8001, "park", map[string]string{"name": "rel 8001"}},
|
||||||
|
{"osm_roads", 8009, "residential", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSingleNodeWaysNotInserted(t *testing.T) {
|
||||||
|
// Ways with single/duplicate nodes are not inserted.
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", 30001, Missing, nil},
|
||||||
|
{"osm_landusages", 30002, Missing, nil},
|
||||||
|
{"osm_landusages", 30003, Missing, nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPolygonWithDuplicateNodesIsValid(t *testing.T) {
|
||||||
|
// Polygon with duplicate nodes is valid.
|
||||||
|
assertValid(t, checkElem{"osm_landusages", 30005, "park", nil})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIncompletePolygons(t *testing.T) {
|
||||||
|
// Non-closed/incomplete polygons are not inserted.
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", 30004, Missing, nil},
|
||||||
|
{"osm_landusages", 30006, Missing, nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResidentialToSecondary(t *testing.T) {
|
||||||
|
// Residential road is not in roads_gen0/1.
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_roads", 40001, "residential", nil},
|
||||||
|
{"osm_roads_gen0", 40001, Missing, nil},
|
||||||
|
{"osm_roads_gen1", 40002, Missing, nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelationBeforeRemove(t *testing.T) {
|
||||||
|
// Relation and way is inserted.
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_buildings", 50011, "yes", nil},
|
||||||
|
{"osm_landusages", -50021, "park", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelationWithoutTags(t *testing.T) {
|
||||||
|
// Relation without tags is inserted.
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_buildings", 50111, Missing, nil},
|
||||||
|
{"osm_buildings", -50121, "yes", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDuplicateIds(t *testing.T) {
|
||||||
|
// Relation/way with same ID is inserted.
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_buildings", 51001, "way", nil},
|
||||||
|
{"osm_buildings", -51001, "mp", nil},
|
||||||
|
{"osm_buildings", 51011, "way", nil},
|
||||||
|
{"osm_buildings", -51011, "mp", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelationUpdatedByNode(t *testing.T) {
|
||||||
|
// Relations was updated after modified node.
|
||||||
|
|
||||||
|
assertArea(t, checkElem{"osm_buildings", -52121, "yes", nil}, 13653930440.868315)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGeneralizedBananaPolygonIsValid(t *testing.T) {
|
||||||
|
// Generalized polygons are valid.
|
||||||
|
|
||||||
|
assertValid(t, checkElem{"osm_landusages", 7101, Missing, nil})
|
||||||
|
// simplified geometies are valid too
|
||||||
|
assertValid(t, checkElem{"osm_landusages_gen0", 7101, Missing, nil})
|
||||||
|
assertValid(t, checkElem{"osm_landusages_gen1", 7101, Missing, nil})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGeneralizedLinestringIsValid(t *testing.T) {
|
||||||
|
// Generalized linestring is valid.
|
||||||
|
|
||||||
|
// geometry is not simple, but valid
|
||||||
|
assertLength(t, checkElem{"osm_roads", 7201, "primary", nil}, 1243660.044819)
|
||||||
|
if ts.g.IsSimple(ts.queryGeom(t, "osm_roads", 7201)) {
|
||||||
|
t.Errorf("expected non-simple geometry for 7201")
|
||||||
|
}
|
||||||
|
// check that geometry 'survives' simplification
|
||||||
|
assertLength(t, checkElem{"osm_roads_gen0", 7201, "primary", nil}, 1243660.044819)
|
||||||
|
assertLength(t, checkElem{"osm_roads_gen1", 7201, "primary", nil}, 1243660.044819)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRingWithGap(t *testing.T) {
|
||||||
|
// Multipolygon and way with gap (overlapping but different endpoints) gets closed
|
||||||
|
assertValid(t, checkElem{"osm_landusages", -7301, Missing, nil})
|
||||||
|
assertValid(t, checkElem{"osm_landusages", 7311, Missing, nil})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMultipolygonWithOpenRing(t *testing.T) {
|
||||||
|
// Multipolygon is inserted even if there is an open ring/member
|
||||||
|
assertValid(t, checkElem{"osm_landusages", -7401, Missing, nil})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUpdatedNodes1(t *testing.T) {
|
||||||
|
// Zig-Zag line is inserted.
|
||||||
|
assertLength(t, checkElem{"osm_roads", 60000, Missing, nil}, 14035.61150207768)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUpdateNodeToCoord1(t *testing.T) {
|
||||||
|
// Node is inserted with tag.
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_amenities", 70001, "police", nil},
|
||||||
|
{"osm_amenities", 70002, Missing, nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnumerateKey(t *testing.T) {
|
||||||
|
// Enumerate from key.
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", 100001, "park", map[string]string{"enum": "1"}},
|
||||||
|
{"osm_landusages", 100002, "park", map[string]string{"enum": "0"}},
|
||||||
|
{"osm_landusages", 100003, "wood", map[string]string{"enum": "15"}},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUpdate(t *testing.T) {
|
||||||
|
ts.updateOsm(t, "./build/complete_db.osc.gz")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNoDuplicates(t *testing.T) {
|
||||||
|
// Relations/ways are only inserted once Checks #66
|
||||||
|
|
||||||
|
for _, table := range []string{"osm_roads", "osm_landusages"} {
|
||||||
|
rows, err := ts.db.Query(
|
||||||
|
fmt.Sprintf(`SELECT osm_id, count(osm_id) FROM "%s"."%s" GROUP BY osm_id HAVING count(osm_id) > 1`,
|
||||||
|
dbschemaProduction, table))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var osmId, count int64
|
||||||
|
for rows.Next() {
|
||||||
|
if err := rows.Scan(&osmId, &count); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if table == "osm_roads" && osmId == 18001 {
|
||||||
|
// # duplicate for TestNodeWayInsertedTwice is expected
|
||||||
|
if count != 2 {
|
||||||
|
t.Error("highway not inserted twice", osmId, count)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
t.Error("found duplicate way in osm_roads", osmId, count)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUpdatedLandusage(t *testing.T) {
|
||||||
|
// Multipolygon relation was modified
|
||||||
|
|
||||||
|
nd := element.Node{Long: 13.4, Lat: 47.5}
|
||||||
|
proj.NodeToMerc(&nd)
|
||||||
|
point, err := geom.Point(ts.g, nd)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
poly := ts.queryGeom(t, "osm_landusages", -1001)
|
||||||
|
// point not in polygon after update
|
||||||
|
if ts.g.Intersects(point, poly) {
|
||||||
|
t.Error("point intersects polygon")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPartialDelete(t *testing.T) {
|
||||||
|
// Deleted relation but nodes are still cached
|
||||||
|
|
||||||
|
cache := ts.cache(t)
|
||||||
|
defer cache.Close()
|
||||||
|
assertCachedNode(t, cache, 2001)
|
||||||
|
assertCachedWay(t, cache, 2001)
|
||||||
|
assertCachedWay(t, cache, 2002)
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", -2001, Missing, nil},
|
||||||
|
{"osm_landusages", 2001, Missing, nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUpdatedNodes(t *testing.T) {
|
||||||
|
// Nodes were added, modified or deleted
|
||||||
|
|
||||||
|
c := ts.cache(t)
|
||||||
|
defer c.Close()
|
||||||
|
if _, err := c.Coords.GetCoord(10000); err != cache.NotFound {
|
||||||
|
t.Fatal("coord not missing")
|
||||||
|
}
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_places", 10001, "village", map[string]string{"name": "Bar"}},
|
||||||
|
{"osm_places", 10002, "city", map[string]string{"name": "Baz"}},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLandusageToWaterarea2(t *testing.T) {
|
||||||
|
// Parks converted to water moved from landusages to waterareas
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_waterareas", 11001, "water", nil},
|
||||||
|
{"osm_waterareas", -12001, "water", nil},
|
||||||
|
{"osm_waterareas", -13001, "water", nil},
|
||||||
|
|
||||||
|
{"osm_waterareas_gen0", 11001, "water", nil},
|
||||||
|
{"osm_waterareas_gen0", -12001, "water", nil},
|
||||||
|
{"osm_waterareas_gen0", -13001, "water", nil},
|
||||||
|
|
||||||
|
{"osm_waterareas_gen1", 11001, "water", nil},
|
||||||
|
{"osm_waterareas_gen1", -12001, "water", nil},
|
||||||
|
{"osm_waterareas_gen1", -13001, "water", nil},
|
||||||
|
|
||||||
|
{"osm_landusages", 11001, Missing, nil},
|
||||||
|
{"osm_landusages", -12001, Missing, nil},
|
||||||
|
{"osm_landusages", -13001, Missing, nil},
|
||||||
|
|
||||||
|
{"osm_landusages_gen0", 11001, Missing, nil},
|
||||||
|
{"osm_landusages_gen0", -12001, Missing, nil},
|
||||||
|
{"osm_landusages_gen0", -13001, Missing, nil},
|
||||||
|
|
||||||
|
{"osm_landusages_gen1", 11001, Missing, nil},
|
||||||
|
{"osm_landusages_gen1", -12001, Missing, nil},
|
||||||
|
{"osm_landusages_gen1", -13001, Missing, nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestChangedHoleTags2(t *testing.T) {
|
||||||
|
// Newly tagged hole is inserted
|
||||||
|
|
||||||
|
cache := ts.cache(t)
|
||||||
|
defer cache.Close()
|
||||||
|
assertCachedWay(t, cache, 14001)
|
||||||
|
assertCachedWay(t, cache, 14011)
|
||||||
|
|
||||||
|
assertArea(t, checkElem{"osm_waterareas", 14011, "water", nil}, 26672019779)
|
||||||
|
assertArea(t, checkElem{"osm_landusages", -14001, "park", nil}, 10373697182)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSplitOuterMultipolygonWay2(t *testing.T) {
|
||||||
|
// Splitted outer way of multipolygon was inserted
|
||||||
|
|
||||||
|
diffCache := ts.diffCache(t)
|
||||||
|
defer diffCache.Close()
|
||||||
|
if ids := diffCache.Ways.Get(15001); len(ids) != 1 || ids[0] != 15001 {
|
||||||
|
t.Error("way does not references relation")
|
||||||
|
}
|
||||||
|
if ids := diffCache.Ways.Get(15002); len(ids) != 1 || ids[0] != 15001 {
|
||||||
|
t.Error("way does not references relation")
|
||||||
|
}
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", 15001, Missing, nil},
|
||||||
|
{"osm_roads", 15002, "residential", nil},
|
||||||
|
})
|
||||||
|
assertArea(t, checkElem{"osm_landusages", -15001, "park", nil}, 9816216452)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMergeOuterMultipolygonWay2(t *testing.T) {
|
||||||
|
// Merged outer way of multipolygon was inserted
|
||||||
|
|
||||||
|
diffCache := ts.diffCache(t)
|
||||||
|
defer diffCache.Close()
|
||||||
|
if ids := diffCache.Ways.Get(16001); len(ids) != 1 || ids[0] != 16001 {
|
||||||
|
t.Error("way does not references relation")
|
||||||
|
}
|
||||||
|
if ids := diffCache.Ways.Get(16002); len(ids) != 0 {
|
||||||
|
t.Error("way references relation")
|
||||||
|
}
|
||||||
|
|
||||||
|
cache := ts.cache(t)
|
||||||
|
defer cache.Close()
|
||||||
|
rel, err := cache.Relations.GetRelation(16001)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(rel.Members) != 2 || rel.Members[0].Id != 16001 || rel.Members[1].Id != 16011 {
|
||||||
|
t.Error("unexpected relation members", rel)
|
||||||
|
}
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", 16001, Missing, nil},
|
||||||
|
{"osm_roads", 16002, Missing, nil},
|
||||||
|
})
|
||||||
|
assertArea(t, checkElem{"osm_landusages", -16001, "park", nil}, 12779350582)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNodeWayRefAfterDelete2(t *testing.T) {
|
||||||
|
// Node does not referece deleted way
|
||||||
|
|
||||||
|
diffCache := ts.diffCache(t)
|
||||||
|
defer diffCache.Close()
|
||||||
|
if ids := diffCache.Coords.Get(20001); len(ids) != 0 {
|
||||||
|
t.Error("node references way")
|
||||||
|
}
|
||||||
|
c := ts.cache(t)
|
||||||
|
defer c.Close()
|
||||||
|
_, err := c.Coords.GetCoord(20002)
|
||||||
|
if err != cache.NotFound {
|
||||||
|
t.Error("found deleted node")
|
||||||
|
}
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_roads", 20001, Missing, nil},
|
||||||
|
{"osm_barrierpoints", 20001, "block", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWayRelRefAfterDelete2(t *testing.T) {
|
||||||
|
// Way does not referece deleted relation
|
||||||
|
|
||||||
|
diffCache := ts.diffCache(t)
|
||||||
|
defer diffCache.Close()
|
||||||
|
if ids := diffCache.Ways.Get(21001); len(ids) != 0 {
|
||||||
|
t.Error("way references relation")
|
||||||
|
}
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_roads", 21001, "residential", nil},
|
||||||
|
{"osm_landusages", 21001, Missing, nil},
|
||||||
|
{"osm_landusages", -21001, Missing, nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResidentialToSecondary2(t *testing.T) {
|
||||||
|
// New secondary (from residential) is now in roads_gen0/1.
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_roads", 40001, "secondary", nil},
|
||||||
|
{"osm_roads_gen0", 40001, "secondary", nil},
|
||||||
|
{"osm_roads_gen1", 40001, "secondary", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelationAfterRemove(t *testing.T) {
|
||||||
|
// Relation is deleted and way is still present.
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_buildings", 50011, "yes", nil},
|
||||||
|
{"osm_landusages", 50021, Missing, nil},
|
||||||
|
{"osm_landusages", -50021, Missing, nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelationWithoutTags2(t *testing.T) {
|
||||||
|
// Relation without tags is removed.
|
||||||
|
|
||||||
|
c := ts.cache(t)
|
||||||
|
defer c.Close()
|
||||||
|
assertCachedWay(t, c, 50111)
|
||||||
|
|
||||||
|
_, err := c.Ways.GetWay(20002)
|
||||||
|
if err != cache.NotFound {
|
||||||
|
t.Error("found deleted node")
|
||||||
|
}
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_buildings", 50111, "yes", nil},
|
||||||
|
{"osm_buildings", 50121, Missing, nil},
|
||||||
|
{"osm_buildings", -50121, Missing, nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDuplicateIds2(t *testing.T) {
|
||||||
|
// Only relation/way with same ID was deleted.
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_buildings", 51001, "way", nil},
|
||||||
|
{"osm_buildings", -51001, Missing, nil},
|
||||||
|
{"osm_buildings", 51011, Missing, nil},
|
||||||
|
{"osm_buildings", -51011, "mp", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelationUpdatedByNode2(t *testing.T) {
|
||||||
|
// Relations was updated after modified node.
|
||||||
|
|
||||||
|
assertArea(t, checkElem{"osm_buildings", -52121, "yes", nil}, 16276875196.653734)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUpdatedWay2(t *testing.T) {
|
||||||
|
// All nodes of straightened way are updated.
|
||||||
|
|
||||||
|
// new length 0.1 degree
|
||||||
|
assertLength(t, checkElem{"osm_roads", 60000, "park", nil}, 20037508.342789244/180.0/10.0)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUpdateNodeToCoord2(t *testing.T) {
|
||||||
|
// Node is becomes coord after tags are removed.
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_amenities", 70001, Missing, nil},
|
||||||
|
{"osm_amenities", 70002, "police", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNoDuplicateInsert(t *testing.T) {
|
||||||
|
// Relation is not inserted again if a nother relation with the same way was modified
|
||||||
|
// Checks #65
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", -201191, "park", nil},
|
||||||
|
{"osm_landusages", -201192, "forest", nil},
|
||||||
|
{"osm_roads", 201151, "residential", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUnsupportedRelation(t *testing.T) {
|
||||||
|
// Unsupported relation type is not inserted with update
|
||||||
|
|
||||||
|
assertRecords(t, []checkElem{
|
||||||
|
{"osm_landusages", -201291, Missing, nil},
|
||||||
|
{"osm_landusages", 201251, "park", nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// #######################################################################
|
||||||
|
|
||||||
|
func TestDeployRevert(t *testing.T) {
|
||||||
|
if ts.tableExists(t, dbschemaImport, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
if !ts.tableExists(t, dbschemaProduction, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads does not exists in schema %s", dbschemaProduction)
|
||||||
|
}
|
||||||
|
if ts.tableExists(t, dbschemaBackup, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads exists in schema %s", dbschemaBackup)
|
||||||
|
}
|
||||||
|
|
||||||
|
ts.importOsm(t)
|
||||||
|
|
||||||
|
if !ts.tableExists(t, dbschemaImport, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads does not exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
if !ts.tableExists(t, dbschemaProduction, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads does not exists in schema %s", dbschemaProduction)
|
||||||
|
}
|
||||||
|
if ts.tableExists(t, dbschemaBackup, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads exists in schema %s", dbschemaBackup)
|
||||||
|
}
|
||||||
|
|
||||||
|
ts.deployOsm(t)
|
||||||
|
|
||||||
|
if ts.tableExists(t, dbschemaImport, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
if !ts.tableExists(t, dbschemaProduction, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads does not exists in schema %s", dbschemaProduction)
|
||||||
|
}
|
||||||
|
if !ts.tableExists(t, dbschemaBackup, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads does exists in schema %s", dbschemaBackup)
|
||||||
|
}
|
||||||
|
|
||||||
|
ts.revertDeployOsm(t)
|
||||||
|
|
||||||
|
if !ts.tableExists(t, dbschemaImport, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads does not exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
if !ts.tableExists(t, dbschemaProduction, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads does not exists in schema %s", dbschemaProduction)
|
||||||
|
}
|
||||||
|
if ts.tableExists(t, dbschemaBackup, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads exists in schema %s", dbschemaBackup)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRemoveBackup(t *testing.T) {
|
||||||
|
if !ts.tableExists(t, dbschemaImport, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads does not exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
if !ts.tableExists(t, dbschemaProduction, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads does not exists in schema %s", dbschemaProduction)
|
||||||
|
}
|
||||||
|
if ts.tableExists(t, dbschemaBackup, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads exists in schema %s", dbschemaBackup)
|
||||||
|
}
|
||||||
|
|
||||||
|
ts.deployOsm(t)
|
||||||
|
|
||||||
|
if ts.tableExists(t, dbschemaImport, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
if !ts.tableExists(t, dbschemaProduction, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads does not exists in schema %s", dbschemaProduction)
|
||||||
|
}
|
||||||
|
if !ts.tableExists(t, dbschemaBackup, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads does exists in schema %s", dbschemaBackup)
|
||||||
|
}
|
||||||
|
|
||||||
|
ts.removeBackupOsm(t)
|
||||||
|
|
||||||
|
if ts.tableExists(t, dbschemaImport, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
if !ts.tableExists(t, dbschemaProduction, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads does not exists in schema %s", dbschemaProduction)
|
||||||
|
}
|
||||||
|
if ts.tableExists(t, dbschemaBackup, "osm_roads") {
|
||||||
|
t.Fatalf("table osm_roads exists in schema %s", dbschemaBackup)
|
||||||
|
}
|
||||||
|
}
|
258
test/helper.py
258
test/helper.py
|
@ -1,258 +0,0 @@
|
||||||
import math
|
|
||||||
import tempfile
|
|
||||||
import shutil
|
|
||||||
import subprocess
|
|
||||||
import psycopg2
|
|
||||||
import psycopg2.extras
|
|
||||||
import json
|
|
||||||
from shapely.wkb import loads as wkb_loads
|
|
||||||
from shapely.geometry import Point
|
|
||||||
import binascii
|
|
||||||
|
|
||||||
import unittest
|
|
||||||
|
|
||||||
__all__ = [
|
|
||||||
"assert_almost_equal",
|
|
||||||
"query_row",
|
|
||||||
"cache_query",
|
|
||||||
"merc_point",
|
|
||||||
"imposm3_import",
|
|
||||||
"imposm3_deploy",
|
|
||||||
"imposm3_update",
|
|
||||||
"imposm3_revert_deploy",
|
|
||||||
"imposm3_remove_backups",
|
|
||||||
"table_exists",
|
|
||||||
"drop_schemas",
|
|
||||||
"TEST_SCHEMA_IMPORT",
|
|
||||||
"TEST_SCHEMA_PRODUCTION",
|
|
||||||
"TEST_SCHEMA_BACKUP",
|
|
||||||
"db_conf",
|
|
||||||
"assert_missing_node",
|
|
||||||
"assert_cached_node",
|
|
||||||
"assert_cached_way",
|
|
||||||
]
|
|
||||||
|
|
||||||
class Dummy(unittest.TestCase):
|
|
||||||
def nop():
|
|
||||||
pass
|
|
||||||
_t = Dummy('nop')
|
|
||||||
assert_almost_equal = _t.assertAlmostEqual
|
|
||||||
|
|
||||||
tmpdir = None
|
|
||||||
|
|
||||||
def setup():
|
|
||||||
global tmpdir
|
|
||||||
tmpdir = tempfile.mkdtemp()
|
|
||||||
|
|
||||||
def teardown():
|
|
||||||
shutil.rmtree(tmpdir)
|
|
||||||
drop_schemas()
|
|
||||||
_close_test_connection(db_conf)
|
|
||||||
|
|
||||||
|
|
||||||
db_conf = {
|
|
||||||
'host': 'localhost',
|
|
||||||
}
|
|
||||||
|
|
||||||
TEST_SCHEMA_IMPORT = "imposm3testimport"
|
|
||||||
TEST_SCHEMA_PRODUCTION = "imposm3testpublic"
|
|
||||||
TEST_SCHEMA_BACKUP = "imposm3testbackup"
|
|
||||||
|
|
||||||
def merc_point(lon, lat):
|
|
||||||
pole = 6378137 * math.pi # 20037508.342789244
|
|
||||||
|
|
||||||
x = lon * pole / 180.0
|
|
||||||
y = math.log(math.tan((90.0+lat)*math.pi/360.0)) / math.pi * pole
|
|
||||||
return Point(x, y)
|
|
||||||
|
|
||||||
|
|
||||||
def pg_db_url(db_conf):
|
|
||||||
return 'postgis://%(host)s' % db_conf
|
|
||||||
|
|
||||||
def create_geom_in_row(rowdict):
|
|
||||||
if rowdict:
|
|
||||||
rowdict['geometry'] = wkb_loads(binascii.unhexlify(rowdict['geometry']))
|
|
||||||
return rowdict
|
|
||||||
|
|
||||||
def query_row(db_conf, table, osmid):
|
|
||||||
conn = _test_connection(db_conf)
|
|
||||||
cur = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)
|
|
||||||
cur.execute('select * from %s.%s where osm_id = %%s' % (TEST_SCHEMA_PRODUCTION, table), [osmid])
|
|
||||||
results = []
|
|
||||||
for row in cur.fetchall():
|
|
||||||
create_geom_in_row(row)
|
|
||||||
results.append(row)
|
|
||||||
cur.close()
|
|
||||||
|
|
||||||
if not results:
|
|
||||||
return None
|
|
||||||
if len(results) == 1:
|
|
||||||
return results[0]
|
|
||||||
return results
|
|
||||||
|
|
||||||
def query_duplicates(db_conf, table):
|
|
||||||
conn = _test_connection(db_conf)
|
|
||||||
cur = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)
|
|
||||||
cur.execute('select osm_id, count(osm_id) from %s.%s group by osm_id having count(osm_id) > 1' % (TEST_SCHEMA_PRODUCTION, table))
|
|
||||||
results = []
|
|
||||||
for row in cur.fetchall():
|
|
||||||
results.append(row)
|
|
||||||
cur.close()
|
|
||||||
return results
|
|
||||||
|
|
||||||
def imposm3_import(db_conf, pbf, mapping_file):
|
|
||||||
_close_test_connection(db_conf)
|
|
||||||
conn = pg_db_url(db_conf)
|
|
||||||
|
|
||||||
try:
|
|
||||||
print subprocess.check_output((
|
|
||||||
"../imposm3 import -connection %s -read %s"
|
|
||||||
" -write"
|
|
||||||
" -cachedir %s"
|
|
||||||
" -diff"
|
|
||||||
" -overwritecache"
|
|
||||||
" -dbschema-import " + TEST_SCHEMA_IMPORT +
|
|
||||||
" -optimize"
|
|
||||||
" -mapping %s ") % (
|
|
||||||
conn, pbf, tmpdir, mapping_file
|
|
||||||
), shell=True)
|
|
||||||
except subprocess.CalledProcessError, ex:
|
|
||||||
print ex.output
|
|
||||||
raise
|
|
||||||
|
|
||||||
def imposm3_deploy(db_conf, mapping_file):
|
|
||||||
_close_test_connection(db_conf)
|
|
||||||
conn = pg_db_url(db_conf)
|
|
||||||
|
|
||||||
try:
|
|
||||||
print subprocess.check_output((
|
|
||||||
"../imposm3 import -connection %s"
|
|
||||||
" -dbschema-import " + TEST_SCHEMA_IMPORT +
|
|
||||||
" -dbschema-production " + TEST_SCHEMA_PRODUCTION +
|
|
||||||
" -dbschema-backup " + TEST_SCHEMA_BACKUP +
|
|
||||||
" -deployproduction"
|
|
||||||
" -mapping %s ") % (
|
|
||||||
conn, mapping_file,
|
|
||||||
), shell=True)
|
|
||||||
except subprocess.CalledProcessError, ex:
|
|
||||||
print ex.output
|
|
||||||
raise
|
|
||||||
|
|
||||||
def imposm3_revert_deploy(db_conf, mapping_file):
|
|
||||||
_close_test_connection(db_conf)
|
|
||||||
conn = pg_db_url(db_conf)
|
|
||||||
|
|
||||||
try:
|
|
||||||
print subprocess.check_output((
|
|
||||||
"../imposm3 import -connection %s"
|
|
||||||
" -dbschema-import " + TEST_SCHEMA_IMPORT +
|
|
||||||
" -dbschema-production " + TEST_SCHEMA_PRODUCTION +
|
|
||||||
" -dbschema-backup " + TEST_SCHEMA_BACKUP +
|
|
||||||
" -revertdeploy"
|
|
||||||
" -mapping %s ") % (
|
|
||||||
conn, mapping_file,
|
|
||||||
), shell=True)
|
|
||||||
except subprocess.CalledProcessError, ex:
|
|
||||||
print ex.output
|
|
||||||
raise
|
|
||||||
|
|
||||||
def imposm3_remove_backups(db_conf, mapping_file):
|
|
||||||
_close_test_connection(db_conf)
|
|
||||||
conn = pg_db_url(db_conf)
|
|
||||||
|
|
||||||
try:
|
|
||||||
print subprocess.check_output((
|
|
||||||
"../imposm3 import -connection %s"
|
|
||||||
" -dbschema-backup " + TEST_SCHEMA_BACKUP +
|
|
||||||
" -removebackup"
|
|
||||||
" -mapping %s ") % (
|
|
||||||
conn, mapping_file,
|
|
||||||
), shell=True)
|
|
||||||
except subprocess.CalledProcessError, ex:
|
|
||||||
print ex.output
|
|
||||||
raise
|
|
||||||
|
|
||||||
def imposm3_update(db_conf, osc, mapping_file):
|
|
||||||
_close_test_connection(db_conf)
|
|
||||||
conn = pg_db_url(db_conf)
|
|
||||||
|
|
||||||
try:
|
|
||||||
print subprocess.check_output((
|
|
||||||
"../imposm3 diff -connection %s"
|
|
||||||
" -cachedir %s"
|
|
||||||
" -limitto clipping.geojson"
|
|
||||||
" -dbschema-production " + TEST_SCHEMA_PRODUCTION +
|
|
||||||
" -mapping %s %s") % (
|
|
||||||
conn, tmpdir, mapping_file, osc,
|
|
||||||
), shell=True)
|
|
||||||
except subprocess.CalledProcessError, ex:
|
|
||||||
print ex.output
|
|
||||||
raise
|
|
||||||
|
|
||||||
def cache_query(nodes='', ways='', relations='', deps='', full=''):
|
|
||||||
if nodes:
|
|
||||||
nodes = '-node ' + ','.join(map(str, nodes))
|
|
||||||
if ways:
|
|
||||||
ways = '-way ' + ','.join(map(str, ways))
|
|
||||||
if relations:
|
|
||||||
relations = '-rel ' + ','.join(map(str, relations))
|
|
||||||
if deps:
|
|
||||||
deps = '-deps'
|
|
||||||
if full:
|
|
||||||
full = '-full'
|
|
||||||
out = subprocess.check_output(
|
|
||||||
"../imposm3 query-cache -cachedir %s %s %s %s %s %s" % (
|
|
||||||
tmpdir, nodes, ways, relations, deps, full),
|
|
||||||
shell=True)
|
|
||||||
print out
|
|
||||||
return json.loads(out)
|
|
||||||
|
|
||||||
def _test_connection(db_conf):
|
|
||||||
if '_connection' in db_conf:
|
|
||||||
return db_conf['_connection']
|
|
||||||
db_conf['_connection'] = psycopg2.connect(**db_conf)
|
|
||||||
return db_conf['_connection']
|
|
||||||
|
|
||||||
def _close_test_connection(db_conf):
|
|
||||||
if '_connection' in db_conf:
|
|
||||||
db_conf['_connection'].close()
|
|
||||||
del db_conf['_connection']
|
|
||||||
|
|
||||||
def table_exists(table, schema=TEST_SCHEMA_IMPORT):
|
|
||||||
conn = _test_connection(db_conf)
|
|
||||||
cur = conn.cursor()
|
|
||||||
cur.execute("SELECT EXISTS(SELECT * FROM information_schema.tables WHERE table_name='%s' AND table_schema='%s')"
|
|
||||||
% (table, schema))
|
|
||||||
|
|
||||||
exists = cur.fetchone()[0]
|
|
||||||
cur.close()
|
|
||||||
return exists
|
|
||||||
|
|
||||||
def assert_missing_node(id):
|
|
||||||
data = cache_query(nodes=[id])
|
|
||||||
if data['nodes'][str(id)]:
|
|
||||||
raise AssertionError('node %d found' % id)
|
|
||||||
|
|
||||||
def assert_cached_node(id, (lon, lat)=(None, None)):
|
|
||||||
data = cache_query(nodes=[id])
|
|
||||||
node = data['nodes'][str(id)]
|
|
||||||
if not node:
|
|
||||||
raise AssertionError('node %d not found' % id)
|
|
||||||
|
|
||||||
if lon and lat:
|
|
||||||
assert_almost_equal(lon, node['lon'], 6)
|
|
||||||
assert_almost_equal(lat, node['lat'], 6)
|
|
||||||
|
|
||||||
def assert_cached_way(id):
|
|
||||||
data = cache_query(ways=[id])
|
|
||||||
if not data['ways'][str(id)]:
|
|
||||||
raise AssertionError('way %d not found' % id)
|
|
||||||
|
|
||||||
def drop_schemas():
|
|
||||||
conn = _test_connection(db_conf)
|
|
||||||
cur = conn.cursor()
|
|
||||||
cur.execute("DROP SCHEMA IF EXISTS %s CASCADE" % TEST_SCHEMA_IMPORT)
|
|
||||||
cur.execute("DROP SCHEMA IF EXISTS %s CASCADE" % TEST_SCHEMA_PRODUCTION)
|
|
||||||
cur.execute("DROP SCHEMA IF EXISTS %s CASCADE" % TEST_SCHEMA_BACKUP)
|
|
||||||
conn.commit()
|
|
||||||
|
|
|
@ -0,0 +1,469 @@
|
||||||
|
package test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"database/sql"
|
||||||
|
"fmt"
|
||||||
|
"log"
|
||||||
|
"math"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/omniscale/imposm3/element"
|
||||||
|
|
||||||
|
"github.com/omniscale/imposm3/cache"
|
||||||
|
|
||||||
|
"github.com/lib/pq/hstore"
|
||||||
|
|
||||||
|
"github.com/omniscale/imposm3/diff"
|
||||||
|
"github.com/omniscale/imposm3/geom/geos"
|
||||||
|
|
||||||
|
"github.com/omniscale/imposm3/config"
|
||||||
|
"github.com/omniscale/imposm3/import_"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
dbschemaImport = "imposm3testimport"
|
||||||
|
dbschemaProduction = "imposm3testproduction"
|
||||||
|
dbschemaBackup = "imposm3testbackup"
|
||||||
|
)
|
||||||
|
|
||||||
|
type importConfig struct {
|
||||||
|
connection string
|
||||||
|
osmFileName string
|
||||||
|
mappingFileName string
|
||||||
|
cacheDir string
|
||||||
|
verbose bool
|
||||||
|
}
|
||||||
|
|
||||||
|
type importTestSuite struct {
|
||||||
|
dir string
|
||||||
|
config importConfig
|
||||||
|
db *sql.DB
|
||||||
|
g *geos.Geos
|
||||||
|
}
|
||||||
|
|
||||||
|
const Missing = ""
|
||||||
|
|
||||||
|
func (s *importTestSuite) importOsm(t *testing.T) {
|
||||||
|
importArgs := []string{
|
||||||
|
"-connection", s.config.connection,
|
||||||
|
"-read", s.config.osmFileName,
|
||||||
|
"-write",
|
||||||
|
"-cachedir", s.config.cacheDir,
|
||||||
|
"-diff",
|
||||||
|
"-overwritecache",
|
||||||
|
"-dbschema-import", dbschemaImport,
|
||||||
|
// "-optimize",
|
||||||
|
"-mapping", s.config.mappingFileName,
|
||||||
|
"-quiet",
|
||||||
|
"-revertdeploy=false",
|
||||||
|
"-deployproduction=false",
|
||||||
|
"-removebackup=false",
|
||||||
|
}
|
||||||
|
|
||||||
|
config.ParseImport(importArgs)
|
||||||
|
import_.Import()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) deployOsm(t *testing.T) {
|
||||||
|
importArgs := []string{
|
||||||
|
"-read=", // overwrite previous options
|
||||||
|
"-write=false",
|
||||||
|
"-optimize=false",
|
||||||
|
"-revertdeploy=false",
|
||||||
|
"-deployproduction",
|
||||||
|
"-removebackup=false",
|
||||||
|
"-connection", s.config.connection,
|
||||||
|
"-dbschema-import", dbschemaImport,
|
||||||
|
"-dbschema-production", dbschemaProduction,
|
||||||
|
"-dbschema-backup", dbschemaBackup,
|
||||||
|
"-deployproduction",
|
||||||
|
"-mapping", s.config.mappingFileName,
|
||||||
|
"-quiet",
|
||||||
|
}
|
||||||
|
|
||||||
|
config.ParseImport(importArgs)
|
||||||
|
import_.Import()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) revertDeployOsm(t *testing.T) {
|
||||||
|
importArgs := []string{
|
||||||
|
"-read=", // overwrite previous options
|
||||||
|
"-write=false",
|
||||||
|
"-optimize=false",
|
||||||
|
"-revertdeploy",
|
||||||
|
"-deployproduction=false",
|
||||||
|
"-removebackup=false",
|
||||||
|
"-connection", s.config.connection,
|
||||||
|
"-dbschema-import", dbschemaImport,
|
||||||
|
"-dbschema-production", dbschemaProduction,
|
||||||
|
"-dbschema-backup", dbschemaBackup,
|
||||||
|
"-revertdeploy",
|
||||||
|
"-deployproduction=false",
|
||||||
|
"-removebackup=false",
|
||||||
|
"-mapping", s.config.mappingFileName,
|
||||||
|
"-quiet",
|
||||||
|
}
|
||||||
|
|
||||||
|
config.ParseImport(importArgs)
|
||||||
|
import_.Import()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) cache(t *testing.T) *cache.OSMCache {
|
||||||
|
c := cache.NewOSMCache(s.config.cacheDir)
|
||||||
|
if err := c.Open(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return c
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) diffCache(t *testing.T) *cache.DiffCache {
|
||||||
|
c := cache.NewDiffCache(s.config.cacheDir)
|
||||||
|
if err := c.Open(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return c
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) removeBackupOsm(t *testing.T) {
|
||||||
|
importArgs := []string{
|
||||||
|
"-read=", // overwrite previous options
|
||||||
|
"-write=false",
|
||||||
|
"-optimize=false",
|
||||||
|
"-revertdeploy=false",
|
||||||
|
"-deployproduction=false",
|
||||||
|
"-removebackup",
|
||||||
|
"-connection", s.config.connection,
|
||||||
|
"-dbschema-import", dbschemaImport,
|
||||||
|
"-dbschema-production", dbschemaProduction,
|
||||||
|
"-dbschema-backup", dbschemaBackup,
|
||||||
|
"-mapping", s.config.mappingFileName,
|
||||||
|
"-quiet",
|
||||||
|
}
|
||||||
|
|
||||||
|
config.ParseImport(importArgs)
|
||||||
|
import_.Import()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) updateOsm(t *testing.T, diffFile string) {
|
||||||
|
args := []string{
|
||||||
|
"-connection", s.config.connection,
|
||||||
|
"-cachedir", s.config.cacheDir,
|
||||||
|
"-limitto", "clipping.geojson",
|
||||||
|
"-dbschema-production", dbschemaProduction,
|
||||||
|
"-mapping", s.config.mappingFileName,
|
||||||
|
diffFile,
|
||||||
|
}
|
||||||
|
config.ParseDiffImport(args)
|
||||||
|
diff.Diff()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) dropSchemas() {
|
||||||
|
var err error
|
||||||
|
_, err = s.db.Exec(fmt.Sprintf(`DROP SCHEMA IF EXISTS %s CASCADE`, dbschemaImport))
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
_, err = s.db.Exec(fmt.Sprintf(`DROP SCHEMA IF EXISTS %s CASCADE`, dbschemaProduction))
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
_, err = s.db.Exec(fmt.Sprintf(`DROP SCHEMA IF EXISTS %s CASCADE`, dbschemaBackup))
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) tableExists(t *testing.T, schema, table string) bool {
|
||||||
|
row := s.db.QueryRow(fmt.Sprintf(`SELECT EXISTS(SELECT * FROM information_schema.tables WHERE table_name='%s' AND table_schema='%s')`, table, schema))
|
||||||
|
var exists bool
|
||||||
|
if err := row.Scan(&exists); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return exists
|
||||||
|
}
|
||||||
|
|
||||||
|
type record struct {
|
||||||
|
id int
|
||||||
|
name string
|
||||||
|
osmType string
|
||||||
|
wkt string
|
||||||
|
missing bool
|
||||||
|
tags map[string]string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) queryExists(t *testing.T, table string, id int64) bool {
|
||||||
|
row := s.db.QueryRow(fmt.Sprintf(`SELECT EXISTS(SELECT * FROM "%s"."%s" WHERE osm_id=$1)`, dbschemaProduction, table), id)
|
||||||
|
var exists bool
|
||||||
|
if err := row.Scan(&exists); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return exists
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) query(t *testing.T, table string, id int64, keys []string) record {
|
||||||
|
kv := make([]string, len(keys))
|
||||||
|
for i, k := range keys {
|
||||||
|
kv[i] = "'" + k + "', " + k + "::varchar"
|
||||||
|
}
|
||||||
|
columns := strings.Join(kv, ", ")
|
||||||
|
if columns == "" {
|
||||||
|
columns = "''::hstore"
|
||||||
|
} else {
|
||||||
|
columns = "hstore(ARRAY[" + columns + "])"
|
||||||
|
}
|
||||||
|
stmt := fmt.Sprintf(`SELECT osm_id, name, type, ST_AsText(geometry), %s FROM "%s"."%s" WHERE osm_id=$1`, columns, dbschemaProduction, table)
|
||||||
|
row := s.db.QueryRow(stmt, id)
|
||||||
|
r := record{}
|
||||||
|
h := hstore.Hstore{}
|
||||||
|
if err := row.Scan(&r.id, &r.name, &r.osmType, &r.wkt, &h); err != nil {
|
||||||
|
if err == sql.ErrNoRows {
|
||||||
|
r.missing = true
|
||||||
|
} else {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(h.Map) > 0 {
|
||||||
|
r.tags = make(map[string]string)
|
||||||
|
}
|
||||||
|
for k, v := range h.Map {
|
||||||
|
if v.Valid {
|
||||||
|
r.tags[k] = v.String
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) queryTags(t *testing.T, table string, id int64) record {
|
||||||
|
stmt := fmt.Sprintf(`SELECT osm_id, tags FROM "%s"."%s" WHERE osm_id=$1`, dbschemaProduction, table)
|
||||||
|
row := s.db.QueryRow(stmt, id)
|
||||||
|
r := record{}
|
||||||
|
h := hstore.Hstore{}
|
||||||
|
if err := row.Scan(&r.id, &h); err != nil {
|
||||||
|
if err == sql.ErrNoRows {
|
||||||
|
r.missing = true
|
||||||
|
} else {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(h.Map) > 0 {
|
||||||
|
r.tags = make(map[string]string)
|
||||||
|
}
|
||||||
|
for k, v := range h.Map {
|
||||||
|
if v.Valid {
|
||||||
|
r.tags[k] = v.String
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) queryRows(t *testing.T, table string, id int64) []record {
|
||||||
|
rows, err := s.db.Query(fmt.Sprintf(`SELECT osm_id, name, type, ST_AsText(geometry) FROM "%s"."%s" WHERE osm_id=$1 ORDER BY type, name, ST_GeometryType(geometry)`, dbschemaProduction, table), id)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rs := []record{}
|
||||||
|
for rows.Next() {
|
||||||
|
var r record
|
||||||
|
if err := rows.Scan(&r.id, &r.name, &r.osmType, &r.wkt); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rs = append(rs, r)
|
||||||
|
}
|
||||||
|
return rs
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) queryRowsTags(t *testing.T, table string, id int64) []record {
|
||||||
|
rows, err := s.db.Query(fmt.Sprintf(`SELECT osm_id, ST_AsText(geometry), tags FROM "%s"."%s" WHERE osm_id=$1 ORDER BY ST_GeometryType(geometry)`, dbschemaProduction, table), id)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rs := []record{}
|
||||||
|
for rows.Next() {
|
||||||
|
var r record
|
||||||
|
h := hstore.Hstore{}
|
||||||
|
if err := rows.Scan(&r.id, &r.wkt, &h); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(h.Map) > 0 {
|
||||||
|
r.tags = make(map[string]string)
|
||||||
|
}
|
||||||
|
for k, v := range h.Map {
|
||||||
|
if v.Valid {
|
||||||
|
r.tags[k] = v.String
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rs = append(rs, r)
|
||||||
|
}
|
||||||
|
return rs
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) queryGeom(t *testing.T, table string, id int64) *geos.Geom {
|
||||||
|
stmt := fmt.Sprintf(`SELECT osm_id, ST_AsText(geometry) FROM "%s"."%s" WHERE osm_id=$1`, dbschemaProduction, table)
|
||||||
|
row := s.db.QueryRow(stmt, id)
|
||||||
|
r := record{}
|
||||||
|
if err := row.Scan(&r.id, &r.wkt); err != nil {
|
||||||
|
if err == sql.ErrNoRows {
|
||||||
|
r.missing = true
|
||||||
|
} else {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
g := geos.NewGeos()
|
||||||
|
defer g.Finish()
|
||||||
|
geom := g.FromWkt(r.wkt)
|
||||||
|
if geom == nil {
|
||||||
|
t.Fatalf("unable to read WKT for %s", id)
|
||||||
|
}
|
||||||
|
return geom
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *importTestSuite) queryDynamic(t *testing.T, table, where string) []map[string]string {
|
||||||
|
stmt := fmt.Sprintf(`SELECT hstore(r) FROM (SELECT ST_AsText(geometry) AS wkt, * FROM "%s"."%s" WHERE %s) AS r`, dbschemaProduction, table, where)
|
||||||
|
rows, err := s.db.Query(stmt)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
results := []map[string]string{}
|
||||||
|
for rows.Next() {
|
||||||
|
h := hstore.Hstore{}
|
||||||
|
if err := rows.Scan(&h); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
r := make(map[string]string)
|
||||||
|
for k, v := range h.Map {
|
||||||
|
if v.Valid {
|
||||||
|
r[k] = v.String
|
||||||
|
}
|
||||||
|
}
|
||||||
|
results = append(results, r)
|
||||||
|
}
|
||||||
|
return results
|
||||||
|
}
|
||||||
|
|
||||||
|
type checkElem struct {
|
||||||
|
table string
|
||||||
|
id int64
|
||||||
|
osmType string
|
||||||
|
tags map[string]string
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertRecordsMissing(t *testing.T, elems []checkElem) {
|
||||||
|
for _, e := range elems {
|
||||||
|
if ts.queryExists(t, e.table, e.id) {
|
||||||
|
t.Errorf("found %d in %d", e.id, e.table)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertRecords(t *testing.T, elems []checkElem) {
|
||||||
|
for _, e := range elems {
|
||||||
|
keys := make([]string, 0, len(e.tags))
|
||||||
|
for k, _ := range e.tags {
|
||||||
|
keys = append(keys, k)
|
||||||
|
}
|
||||||
|
r := ts.query(t, e.table, e.id, keys)
|
||||||
|
if e.osmType == "" {
|
||||||
|
if r.missing {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
t.Errorf("got unexpected record %d", r.id)
|
||||||
|
}
|
||||||
|
if r.osmType != e.osmType {
|
||||||
|
t.Errorf("got unexpected type %s != %s for %d", r.osmType, e.osmType, e.id)
|
||||||
|
}
|
||||||
|
for k, v := range e.tags {
|
||||||
|
if r.tags[k] != v {
|
||||||
|
t.Errorf("%s does not match for %d %s != %s", k, e.id, r.tags[k], v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertHstore(t *testing.T, elems []checkElem) {
|
||||||
|
for _, e := range elems {
|
||||||
|
r := ts.queryTags(t, e.table, e.id)
|
||||||
|
if e.osmType == "" {
|
||||||
|
if r.missing {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
t.Errorf("got unexpected record %d", r.id)
|
||||||
|
}
|
||||||
|
if len(e.tags) != len(r.tags) {
|
||||||
|
t.Errorf("tags for %d differ %v != %v", e.id, r.tags, e.tags)
|
||||||
|
}
|
||||||
|
for k, v := range e.tags {
|
||||||
|
if r.tags[k] != v {
|
||||||
|
t.Errorf("%s does not match for %d %s != %s", k, e.id, r.tags[k], v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertValid(t *testing.T, e checkElem) {
|
||||||
|
geom := ts.queryGeom(t, e.table, e.id)
|
||||||
|
if !ts.g.IsValid(geom) {
|
||||||
|
t.Fatalf("geometry of %d is invalid", e.id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertArea(t *testing.T, e checkElem, expect float64) {
|
||||||
|
geom := ts.queryGeom(t, e.table, e.id)
|
||||||
|
if !ts.g.IsValid(geom) {
|
||||||
|
t.Fatalf("geometry of %d is invalid", e.id)
|
||||||
|
}
|
||||||
|
actual := geom.Area()
|
||||||
|
if math.Abs(expect-actual) > 1 {
|
||||||
|
t.Errorf("unexpected size of %d %f!=%f", e.id, actual, expect)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertLength(t *testing.T, e checkElem, expect float64) {
|
||||||
|
geom := ts.queryGeom(t, e.table, e.id)
|
||||||
|
if !ts.g.IsValid(geom) {
|
||||||
|
t.Fatalf("geometry of %d is invalid", e.id)
|
||||||
|
}
|
||||||
|
actual := geom.Length()
|
||||||
|
if math.Abs(expect-actual) > 1 {
|
||||||
|
t.Errorf("unexpected size of %d %f!=%f", e.id, actual, expect)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertGeomType(t *testing.T, e checkElem, expect string) {
|
||||||
|
actual := ts.g.Type(ts.queryGeom(t, e.table, e.id))
|
||||||
|
if actual != expect {
|
||||||
|
t.Errorf("expected %s geometry for %d, got %s", expect, e.id, actual)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertCachedWay(t *testing.T, c *cache.OSMCache, id int64) *element.Way {
|
||||||
|
way, err := c.Ways.GetWay(id)
|
||||||
|
if err == cache.NotFound {
|
||||||
|
t.Errorf("missing way %d", id)
|
||||||
|
} else if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if way.Id != id {
|
||||||
|
t.Errorf("cached way contains invalid id, %d != %d", way.Id, id)
|
||||||
|
}
|
||||||
|
return way
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertCachedNode(t *testing.T, c *cache.OSMCache, id int64) *element.Node {
|
||||||
|
node, err := c.Nodes.GetNode(id)
|
||||||
|
if err == cache.NotFound {
|
||||||
|
node, err = c.Coords.GetCoord(id)
|
||||||
|
if err == cache.NotFound {
|
||||||
|
t.Errorf("missing node %d", id)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
} else if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if node.Id != id {
|
||||||
|
t.Errorf("cached node contains invalid id, %d != %d", node.Id, id)
|
||||||
|
}
|
||||||
|
return node
|
||||||
|
}
|
|
@ -0,0 +1,44 @@
|
||||||
|
<?xml version='1.0' encoding='UTF-8'?>
|
||||||
|
<osmChange version="0.6" generator="Osmosis 0.41">
|
||||||
|
<modify>
|
||||||
|
<node id="100103" version="2" timestamp="2015-12-31T23:59:99Z" lat="53.001" lon="8.201"/>
|
||||||
|
|
||||||
|
<way id="100503" version="1" timestamp="2015-12-31T23:59:99Z">
|
||||||
|
<nd ref="100103"/>
|
||||||
|
<nd ref="100104"/>
|
||||||
|
<tag k="name" v="new name"/>
|
||||||
|
</way>
|
||||||
|
</modify>
|
||||||
|
|
||||||
|
<modify>
|
||||||
|
<relation id="100902" version="23" timestamp="2015-06-02T04:13:19Z">
|
||||||
|
<member type="node" ref="100104" role="stop_entry_only"/>
|
||||||
|
<member type="way" ref="100514" role="platform_entry_only"/>
|
||||||
|
<member type="node" ref="100103" role="stop"/>
|
||||||
|
<member type="way" ref="100513" role="platform"/>
|
||||||
|
<member type="node" ref="100102" role="halt"/> <!-- changed role -->
|
||||||
|
<!-- <member type="way" ref="100512" role="platform"/> removed -->
|
||||||
|
<member type="node" ref="100101" role="stop_exit_only"/>
|
||||||
|
<member type="way" ref="100511" role="platform_exit_only"/>
|
||||||
|
<member type="way" ref="100503" role=""/>
|
||||||
|
<member type="way" ref="100502" role=""/>
|
||||||
|
<member type="way" ref="100501" role=""/>
|
||||||
|
<tag k="colour" v="#F5A9E1"/>
|
||||||
|
<tag k="from" v="B"/>
|
||||||
|
<tag k="name" v="Bus 301: B => A"/>
|
||||||
|
<tag k="network" v="ABC"/>
|
||||||
|
<tag k="ref" v="301"/>
|
||||||
|
<tag k="route" v="bus"/>
|
||||||
|
<tag k="to" v="A"/>
|
||||||
|
<tag k="type" v="route"/>
|
||||||
|
</relation>
|
||||||
|
</modify>
|
||||||
|
|
||||||
|
<modify>
|
||||||
|
<!-- modified node updates relation -->
|
||||||
|
<node id="110101" version="1" timestamp="2015-12-31T23:59:99Z" lat="53.0" lon="8.200">
|
||||||
|
<tag k="name" v="Stop2"/>
|
||||||
|
</node>
|
||||||
|
</modify>
|
||||||
|
|
||||||
|
</osmChange>
|
|
@ -0,0 +1,143 @@
|
||||||
|
<?xml version="1.0" encoding="UTF-8"?>
|
||||||
|
<osm version="0.6" generator="CGImap 0.4.0 (2097 thorn-01.openstreetmap.org)" copyright="OpenStreetMap and contributors" attribution="http://www.openstreetmap.org/copyright" license="http://opendatacommons.org/licenses/odbl/1-0/">
|
||||||
|
<node id="100101" version="1" timestamp="2015-12-31T23:59:99Z" lat="53.0" lon="8.200"/>
|
||||||
|
<node id="100102" version="1" timestamp="2015-12-31T23:59:99Z" lat="53.0" lon="8.201"/>
|
||||||
|
<node id="100103" version="1" timestamp="2015-12-31T23:59:99Z" lat="53.0" lon="8.202"/>
|
||||||
|
<node id="100104" version="1" timestamp="2015-12-31T23:59:99Z" lat="53.0" lon="8.203"/>
|
||||||
|
|
||||||
|
<node id="100201" version="1" timestamp="2015-12-31T23:59:99Z" lat="53.0001" lon="8.2000"/>
|
||||||
|
<node id="100211" version="1" timestamp="2015-12-31T23:59:99Z" lat="53.0001" lon="8.2001"/>
|
||||||
|
<node id="100202" version="1" timestamp="2015-12-31T23:59:99Z" lat="53.0001" lon="8.2010"/>
|
||||||
|
<node id="100212" version="1" timestamp="2015-12-31T23:59:99Z" lat="53.0001" lon="8.2011"/>
|
||||||
|
<node id="100203" version="1" timestamp="2015-12-31T23:59:99Z" lat="53.0001" lon="8.2020"/>
|
||||||
|
<node id="100213" version="1" timestamp="2015-12-31T23:59:99Z" lat="53.0001" lon="8.2021"/>
|
||||||
|
<node id="100204" version="1" timestamp="2015-12-31T23:59:99Z" lat="53.0001" lon="8.2030"/>
|
||||||
|
<node id="100214" version="1" timestamp="2015-12-31T23:59:99Z" lat="53.0001" lon="8.2031"/>
|
||||||
|
|
||||||
|
|
||||||
|
<way id="100501" version="1" timestamp="2015-12-31T23:59:99Z">
|
||||||
|
<nd ref="100101"/>
|
||||||
|
<nd ref="100102"/>
|
||||||
|
<tag k="highway" v="residential"/>
|
||||||
|
<tag k="name" v="Residential Street"/>
|
||||||
|
</way>
|
||||||
|
|
||||||
|
<way id="100502" version="1" timestamp="2015-12-31T23:59:99Z">
|
||||||
|
<nd ref="100102"/>
|
||||||
|
<nd ref="100103"/>
|
||||||
|
<tag k="highway" v="residential"/>
|
||||||
|
<tag k="name" v="Residential Street"/>
|
||||||
|
</way>
|
||||||
|
|
||||||
|
<way id="100503" version="1" timestamp="2015-12-31T23:59:99Z">
|
||||||
|
<nd ref="100103"/>
|
||||||
|
<nd ref="100104"/>
|
||||||
|
<!-- untagged -->
|
||||||
|
</way>
|
||||||
|
|
||||||
|
<way id="100511" version="1" timestamp="2015-12-31T23:59:99Z">
|
||||||
|
<nd ref="100201"/>
|
||||||
|
<nd ref="100211"/>
|
||||||
|
</way>
|
||||||
|
|
||||||
|
<way id="100512" version="1" timestamp="2015-12-31T23:59:99Z">
|
||||||
|
<nd ref="100202"/>
|
||||||
|
<nd ref="100212"/>
|
||||||
|
</way>
|
||||||
|
|
||||||
|
<way id="100513" version="1" timestamp="2015-12-31T23:59:99Z">
|
||||||
|
<nd ref="100203"/>
|
||||||
|
<nd ref="100213"/>
|
||||||
|
</way>
|
||||||
|
|
||||||
|
<way id="100514" version="1" timestamp="2015-12-31T23:59:99Z">
|
||||||
|
<nd ref="100204"/>
|
||||||
|
<nd ref="100214"/>
|
||||||
|
</way>
|
||||||
|
|
||||||
|
<relation id="100901" version="23" timestamp="2015-06-02T04:13:19Z">
|
||||||
|
<member type="node" ref="100101" role="stop_entry_only"/>
|
||||||
|
<member type="way" ref="100511" role="platform_entry_only"/>
|
||||||
|
<member type="node" ref="100102" role="stop"/>
|
||||||
|
<member type="way" ref="100512" role="platform"/>
|
||||||
|
<member type="node" ref="100103" role="stop"/>
|
||||||
|
<member type="way" ref="100513" role="platform"/>
|
||||||
|
<member type="node" ref="100104" role="stop_exit_only"/>
|
||||||
|
<member type="way" ref="100514" role="platform_exit_only"/>
|
||||||
|
<member type="way" ref="100501" role=""/>
|
||||||
|
<member type="way" ref="100502" role=""/>
|
||||||
|
<member type="way" ref="100503" role=""/>
|
||||||
|
<tag k="colour" v="#F5A9E1"/>
|
||||||
|
<tag k="from" v="A"/>
|
||||||
|
<tag k="name" v="Bus 301: A => B"/>
|
||||||
|
<tag k="network" v="ABC"/>
|
||||||
|
<tag k="ref" v="301"/>
|
||||||
|
<tag k="route" v="bus"/>
|
||||||
|
<tag k="to" v="B"/>
|
||||||
|
<tag k="type" v="route"/>
|
||||||
|
</relation>
|
||||||
|
|
||||||
|
<relation id="100902" version="23" timestamp="2015-06-02T04:13:19Z">
|
||||||
|
<member type="node" ref="100104" role="stop_entry_only"/>
|
||||||
|
<member type="way" ref="100514" role="platform_entry_only"/>
|
||||||
|
<member type="node" ref="100103" role="stop"/>
|
||||||
|
<member type="way" ref="100513" role="platform"/>
|
||||||
|
<member type="node" ref="100102" role="stop"/>
|
||||||
|
<member type="way" ref="100512" role="platform"/>
|
||||||
|
<member type="node" ref="100101" role="stop_exit_only"/>
|
||||||
|
<member type="way" ref="100511" role="platform_exit_only"/>
|
||||||
|
<member type="way" ref="100503" role=""/>
|
||||||
|
<member type="way" ref="100502" role=""/>
|
||||||
|
<member type="way" ref="100501" role=""/>
|
||||||
|
<tag k="colour" v="#F5A9E1"/>
|
||||||
|
<tag k="from" v="B"/>
|
||||||
|
<tag k="name" v="Bus 301: B => A"/>
|
||||||
|
<tag k="network" v="ABC"/>
|
||||||
|
<tag k="ref" v="301"/>
|
||||||
|
<tag k="route" v="bus"/>
|
||||||
|
<tag k="to" v="A"/>
|
||||||
|
<tag k="type" v="route"/>
|
||||||
|
</relation>
|
||||||
|
|
||||||
|
|
||||||
|
<relation id="100903" version="23" timestamp="2015-06-02T04:13:19Z">
|
||||||
|
<member type="node" ref="100101" role="stop_entry_only"/>
|
||||||
|
<member type="way" ref="100511" role="platform_entry_only"/>
|
||||||
|
<member type="node" ref="100102" role="stop_exit_only"/>
|
||||||
|
<member type="way" ref="100512" role="platform_exit_only"/>
|
||||||
|
<member type="way" ref="100501" role=""/>
|
||||||
|
<tag k="colour" v="#F5A9E1"/>
|
||||||
|
<tag k="from" v="A"/>
|
||||||
|
<tag k="name" v="Bus 302: A => C"/>
|
||||||
|
<tag k="network" v="ABC"/>
|
||||||
|
<tag k="ref" v="301"/>
|
||||||
|
<tag k="route" v="bus"/>
|
||||||
|
<tag k="to" v="C"/>
|
||||||
|
<tag k="type" v="route"/>
|
||||||
|
</relation>
|
||||||
|
|
||||||
|
|
||||||
|
<relation id="100911" version="23" timestamp="2015-06-02T04:13:19Z">
|
||||||
|
<member type="relation" ref="100901" role=""/>
|
||||||
|
<member type="relation" ref="100902" role=""/>
|
||||||
|
<tag k="colour" v="#F5A9E1"/>
|
||||||
|
<tag k="name" v="Bus 301"/>
|
||||||
|
<tag k="network" v="ABC"/>
|
||||||
|
<tag k="ref" v="301"/>
|
||||||
|
<tag k="route_master" v="bus"/>
|
||||||
|
<tag k="type" v="route_master"/>
|
||||||
|
</relation>
|
||||||
|
|
||||||
|
|
||||||
|
<!-- modified node updates relation -->
|
||||||
|
<node id="110101" version="1" timestamp="2015-12-31T23:59:99Z" lat="53.0" lon="8.200">
|
||||||
|
<tag k="name" v="Stop"/>
|
||||||
|
</node>
|
||||||
|
|
||||||
|
<relation id="110901" version="23" timestamp="2015-06-02T04:13:19Z">
|
||||||
|
<member type="node" ref="110101" role="stop"/>
|
||||||
|
<tag k="route" v="bus"/>
|
||||||
|
<tag k="type" v="route"/>
|
||||||
|
</relation>
|
||||||
|
|
||||||
|
</osm>
|
|
@ -0,0 +1,70 @@
|
||||||
|
tags:
|
||||||
|
load_all: true
|
||||||
|
exclude:
|
||||||
|
- created_by
|
||||||
|
- source
|
||||||
|
|
||||||
|
tables:
|
||||||
|
master_routes:
|
||||||
|
type: relation_member
|
||||||
|
fields:
|
||||||
|
- name: osm_id
|
||||||
|
type: id
|
||||||
|
- name: member
|
||||||
|
type: member_id
|
||||||
|
- name: index
|
||||||
|
type: member_index
|
||||||
|
- name: role
|
||||||
|
type: member_role
|
||||||
|
- name: type
|
||||||
|
type: member_type
|
||||||
|
- name: geometry
|
||||||
|
type: geometry
|
||||||
|
- name: subname
|
||||||
|
key: name
|
||||||
|
type: string
|
||||||
|
from_member: true
|
||||||
|
- key: name
|
||||||
|
name: name
|
||||||
|
type: string
|
||||||
|
mapping:
|
||||||
|
route_master: [bus]
|
||||||
|
route_members:
|
||||||
|
type: relation_member
|
||||||
|
fields:
|
||||||
|
- name: osm_id
|
||||||
|
type: id
|
||||||
|
- key: ref
|
||||||
|
name: ref
|
||||||
|
type: string
|
||||||
|
- name: member
|
||||||
|
type: member_id
|
||||||
|
- name: index
|
||||||
|
type: member_index
|
||||||
|
- name: role
|
||||||
|
type: member_role
|
||||||
|
- name: type
|
||||||
|
type: member_type
|
||||||
|
- name: geometry
|
||||||
|
type: geometry
|
||||||
|
- name: relname
|
||||||
|
key: name
|
||||||
|
type: string
|
||||||
|
- name: name
|
||||||
|
key: name
|
||||||
|
type: string
|
||||||
|
from_member: true
|
||||||
|
mapping:
|
||||||
|
route: [bus, tram, rail]
|
||||||
|
routes:
|
||||||
|
type: relation
|
||||||
|
fields:
|
||||||
|
- name: osm_id
|
||||||
|
type: id
|
||||||
|
- key: ref
|
||||||
|
name: ref
|
||||||
|
type: string
|
||||||
|
- name: tags
|
||||||
|
type: hstore_tags
|
||||||
|
mapping:
|
||||||
|
route: [bus, tram, rail]
|
|
@ -0,0 +1,129 @@
|
||||||
|
package test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"database/sql"
|
||||||
|
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/omniscale/imposm3/geom/geos"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRouteRelation_Prepare(t *testing.T) {
|
||||||
|
ts.dir = "/tmp/imposm3test"
|
||||||
|
ts.config = importConfig{
|
||||||
|
connection: "postgis://",
|
||||||
|
cacheDir: ts.dir,
|
||||||
|
osmFileName: "build/route_relation.pbf",
|
||||||
|
mappingFileName: "route_relation_mapping.yml",
|
||||||
|
}
|
||||||
|
ts.g = geos.NewGeos()
|
||||||
|
|
||||||
|
var err error
|
||||||
|
ts.db, err = sql.Open("postgres", "sslmode=disable")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ts.dropSchemas()
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRouteRelation_Import(t *testing.T) {
|
||||||
|
if ts.tableExists(t, dbschemaImport, "osm_routes") != false {
|
||||||
|
t.Fatalf("table osm_routes exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
ts.importOsm(t)
|
||||||
|
if ts.tableExists(t, dbschemaImport, "osm_routes") != true {
|
||||||
|
t.Fatalf("table osm_routes does not exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRouteRelation_Deploy(t *testing.T) {
|
||||||
|
ts.deployOsm(t)
|
||||||
|
if ts.tableExists(t, dbschemaImport, "osm_routes") != false {
|
||||||
|
t.Fatalf("table osm_routes exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
if ts.tableExists(t, dbschemaProduction, "osm_routes") != true {
|
||||||
|
t.Fatalf("table osm_routes does not exists in schema %s", dbschemaProduction)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRouteRelation_RelationData(t *testing.T) {
|
||||||
|
// check tags of relation
|
||||||
|
r := ts.queryTags(t, "osm_routes", -100901)
|
||||||
|
if r.tags["name"] != "Bus 301: A => B" {
|
||||||
|
t.Error(r)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRouteRelation_MemberGeomUpdated1(t *testing.T) {
|
||||||
|
rows := ts.queryDynamic(t, "osm_route_members", "osm_id = -100902 AND member = 100502")
|
||||||
|
if len(rows) != 1 {
|
||||||
|
t.Fatal(rows)
|
||||||
|
}
|
||||||
|
g := ts.g.FromWkt(rows[0]["wkt"])
|
||||||
|
if g.Length() != 111.32448543701321 {
|
||||||
|
t.Fatal(g.Length())
|
||||||
|
}
|
||||||
|
|
||||||
|
rows = ts.queryDynamic(t, "osm_route_members", "osm_id = -100902 AND member = 100503")
|
||||||
|
if len(rows) != 1 {
|
||||||
|
t.Fatal(rows)
|
||||||
|
}
|
||||||
|
if rows[0]["name"] != "" {
|
||||||
|
t.Error(rows[0])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// #######################################################################
|
||||||
|
|
||||||
|
func TestRouteRelation_Update(t *testing.T) {
|
||||||
|
ts.updateOsm(t, "./build/route_relation.osc.gz")
|
||||||
|
}
|
||||||
|
|
||||||
|
// #######################################################################
|
||||||
|
|
||||||
|
func TestRouteRelation_MemberGeomUpdated2(t *testing.T) {
|
||||||
|
rows := ts.queryDynamic(t, "osm_route_members", "osm_id = -100902 AND member = 100502")
|
||||||
|
if len(rows) != 1 {
|
||||||
|
t.Fatal(rows)
|
||||||
|
}
|
||||||
|
g := ts.g.FromWkt(rows[0]["wkt"])
|
||||||
|
if g.Length() != 184.97560221624542 {
|
||||||
|
t.Fatal(g.Length())
|
||||||
|
}
|
||||||
|
|
||||||
|
// tag from member is updated
|
||||||
|
rows = ts.queryDynamic(t, "osm_route_members", "osm_id = -100902 AND member = 100503")
|
||||||
|
if len(rows) != 1 {
|
||||||
|
t.Fatal(rows)
|
||||||
|
}
|
||||||
|
if rows[0]["name"] != "new name" {
|
||||||
|
t.Error(rows[0])
|
||||||
|
}
|
||||||
|
|
||||||
|
// member is removed
|
||||||
|
rows = ts.queryDynamic(t, "osm_route_members", "osm_id = -100902 AND member = 100512")
|
||||||
|
if len(rows) != 0 {
|
||||||
|
t.Fatal(rows)
|
||||||
|
}
|
||||||
|
|
||||||
|
// role from member is updated
|
||||||
|
rows = ts.queryDynamic(t, "osm_route_members", "osm_id = -100902 AND member = 100102")
|
||||||
|
if len(rows) != 1 {
|
||||||
|
t.Fatal(rows)
|
||||||
|
}
|
||||||
|
if rows[0]["role"] != "halt" {
|
||||||
|
t.Error(rows[0])
|
||||||
|
}
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRouteRelation_MemberUpdatedByNode(t *testing.T) {
|
||||||
|
// check that member is updated after node was modified
|
||||||
|
rows := ts.queryDynamic(t, "osm_route_members", "osm_id = -110901 AND member = 110101")
|
||||||
|
if len(rows) != 1 {
|
||||||
|
t.Fatal(rows)
|
||||||
|
}
|
||||||
|
if rows[0]["name"] != "Stop2" {
|
||||||
|
t.Error(rows[0])
|
||||||
|
}
|
||||||
|
}
|
|
@ -0,0 +1,185 @@
|
||||||
|
package test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"database/sql"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/omniscale/imposm3/geom/geos"
|
||||||
|
)
|
||||||
|
|
||||||
|
const RelOffset = -1e17
|
||||||
|
|
||||||
|
func TestSingleTable_Prepare(t *testing.T) {
|
||||||
|
ts.dir = "/tmp/imposm3test"
|
||||||
|
ts.config = importConfig{
|
||||||
|
connection: "postgis://",
|
||||||
|
cacheDir: ts.dir,
|
||||||
|
osmFileName: "build/single_table.pbf",
|
||||||
|
mappingFileName: "single_table_mapping.json",
|
||||||
|
}
|
||||||
|
ts.g = geos.NewGeos()
|
||||||
|
|
||||||
|
var err error
|
||||||
|
ts.db, err = sql.Open("postgres", "sslmode=disable")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ts.dropSchemas()
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSingleTable_Import(t *testing.T) {
|
||||||
|
if ts.tableExists(t, dbschemaImport, "osm_all") != false {
|
||||||
|
t.Fatalf("table osm_all exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
ts.importOsm(t)
|
||||||
|
if ts.tableExists(t, dbschemaImport, "osm_all") != true {
|
||||||
|
t.Fatalf("table osm_all does not exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSingleTable_Deploy(t *testing.T) {
|
||||||
|
ts.deployOsm(t)
|
||||||
|
if ts.tableExists(t, dbschemaImport, "osm_all") != false {
|
||||||
|
t.Fatalf("table osm_all exists in schema %s", dbschemaImport)
|
||||||
|
}
|
||||||
|
if ts.tableExists(t, dbschemaProduction, "osm_all") != true {
|
||||||
|
t.Fatalf("table osm_all does not exists in schema %s", dbschemaProduction)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSingleTable_NonMappedNodeIsMissing(t *testing.T) {
|
||||||
|
// Node without mapped tags is missing.
|
||||||
|
cache := ts.cache(t)
|
||||||
|
defer cache.Close()
|
||||||
|
assertCachedNode(t, cache, 10001)
|
||||||
|
|
||||||
|
assertHstore(t, []checkElem{
|
||||||
|
{"osm_all", 10001, Missing, nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSingleTable_MappedNode(t *testing.T) {
|
||||||
|
// Node is stored with all tags.
|
||||||
|
cache := ts.cache(t)
|
||||||
|
defer cache.Close()
|
||||||
|
assertCachedNode(t, cache, 10002)
|
||||||
|
|
||||||
|
assertHstore(t, []checkElem{
|
||||||
|
{"osm_all", 10002, "*", map[string]string{"random": "tag", "but": "mapped", "poi": "unicorn"}},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSingleTable_NonMappedWayIsMissing(t *testing.T) {
|
||||||
|
// Way without mapped tags is missing.
|
||||||
|
cache := ts.cache(t)
|
||||||
|
defer cache.Close()
|
||||||
|
assertCachedWay(t, cache, 20101)
|
||||||
|
assertCachedWay(t, cache, 20102)
|
||||||
|
assertCachedWay(t, cache, 20103)
|
||||||
|
|
||||||
|
assertHstore(t, []checkElem{
|
||||||
|
{"osm_all", 20101, Missing, nil},
|
||||||
|
{"osm_all", 20102, Missing, nil},
|
||||||
|
{"osm_all", 20103, Missing, nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSingleTable_MappedWay(t *testing.T) {
|
||||||
|
// Way is stored with all tags.
|
||||||
|
cache := ts.cache(t)
|
||||||
|
defer cache.Close()
|
||||||
|
assertCachedWay(t, cache, 20201)
|
||||||
|
|
||||||
|
assertHstore(t, []checkElem{
|
||||||
|
{"osm_all", -20201, "*", map[string]string{"random": "tag", "highway": "yes"}},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSingleTable_NonMappedClosedWayIsMissing(t *testing.T) {
|
||||||
|
// Closed way without mapped tags is missing.
|
||||||
|
cache := ts.cache(t)
|
||||||
|
defer cache.Close()
|
||||||
|
assertCachedWay(t, cache, 20301)
|
||||||
|
assertHstore(t, []checkElem{
|
||||||
|
{"osm_all", 20301, Missing, nil},
|
||||||
|
{"osm_all", -20301, Missing, nil},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSingleTable_MappedClosedWay(t *testing.T) {
|
||||||
|
// Closed way is stored with all tags.
|
||||||
|
assertHstore(t, []checkElem{
|
||||||
|
{"osm_all", -20401, "*", map[string]string{"random": "tag", "building": "yes"}},
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSingleTable_MappedClosedWayAreaYes(t *testing.T) {
|
||||||
|
// Closed way with area=yes is not stored as linestring.
|
||||||
|
assertHstore(t, []checkElem{
|
||||||
|
{"osm_all", -20501, "*", map[string]string{"random": "tag", "landuse": "grass", "highway": "pedestrian", "area": "yes"}},
|
||||||
|
})
|
||||||
|
assertGeomType(t, checkElem{"osm_all", -20501, "*", nil}, "Polygon")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSingleTable_MappedClosedWayAreaNo(t *testing.T) {
|
||||||
|
// Closed way with area=no is not stored as polygon.
|
||||||
|
assertHstore(t, []checkElem{
|
||||||
|
{"osm_all", -20502, "*", map[string]string{"random": "tag", "landuse": "grass", "highway": "pedestrian", "area": "no"}},
|
||||||
|
})
|
||||||
|
assertGeomType(t, checkElem{"osm_all", -20502, "*", nil}, "LineString")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSingleTable_MappedClosedWayWithoutArea(t *testing.T) {
|
||||||
|
// Closed way without area is stored as mapped (linestring and polygon).
|
||||||
|
|
||||||
|
rows := ts.queryRowsTags(t, "osm_all", -20601)
|
||||||
|
if len(rows) != 2 || strings.HasPrefix(rows[0].wkt, "LineString") || strings.HasPrefix(rows[1].wkt, "Polygon") {
|
||||||
|
t.Errorf("unexpected geometries: %v", rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSingleTable_DuplicateIds1(t *testing.T) {
|
||||||
|
// Points/lines/polygons with same ID are inserted.
|
||||||
|
|
||||||
|
assertHstore(t, []checkElem{
|
||||||
|
{"osm_all", 31101, "*", map[string]string{"amenity": "cafe"}},
|
||||||
|
})
|
||||||
|
|
||||||
|
rows := ts.queryRowsTags(t, "osm_all", -31101)
|
||||||
|
if len(rows) != 2 || strings.HasPrefix(rows[0].wkt, "LineString") || strings.HasPrefix(rows[1].wkt, "Polygon") {
|
||||||
|
t.Errorf("unexpected geometries: %v", rows)
|
||||||
|
}
|
||||||
|
|
||||||
|
assertHstore(t, []checkElem{
|
||||||
|
{"osm_all", RelOffset - 31101, "*", map[string]string{"building": "yes"}},
|
||||||
|
})
|
||||||
|
assertGeomType(t, checkElem{"osm_all", RelOffset - 31101, "*", nil}, "Polygon")
|
||||||
|
}
|
||||||
|
|
||||||
|
// #######################################################################
|
||||||
|
|
||||||
|
func TestSingleTable_Update(t *testing.T) {
|
||||||
|
ts.updateOsm(t, "./build/single_table.osc.gz")
|
||||||
|
}
|
||||||
|
|
||||||
|
// #######################################################################
|
||||||
|
|
||||||
|
func TestSingleTable_DuplicateIds2(t *testing.T) {
|
||||||
|
// Node moved and ways/rels with same ID are still present.
|
||||||
|
|
||||||
|
assertHstore(t, []checkElem{
|
||||||
|
{"osm_all", 31101, "*", map[string]string{"amenity": "cafe"}},
|
||||||
|
})
|
||||||
|
|
||||||
|
rows := ts.queryRowsTags(t, "osm_all", -31101)
|
||||||
|
if len(rows) != 2 || strings.HasPrefix(rows[0].wkt, "LineString") || strings.HasPrefix(rows[1].wkt, "Polygon") {
|
||||||
|
t.Errorf("unexpected geometries: %v", rows)
|
||||||
|
}
|
||||||
|
|
||||||
|
assertHstore(t, []checkElem{
|
||||||
|
{"osm_all", RelOffset - 31101, "*", map[string]string{"building": "yes"}},
|
||||||
|
})
|
||||||
|
assertGeomType(t, checkElem{"osm_all", RelOffset - 31101, "*", nil}, "Polygon")
|
||||||
|
}
|
|
@ -1,184 +0,0 @@
|
||||||
import psycopg2
|
|
||||||
import psycopg2.extras
|
|
||||||
|
|
||||||
import helper as t
|
|
||||||
|
|
||||||
psycopg2.extras.register_hstore(psycopg2.connect(**t.db_conf), globally=True)
|
|
||||||
|
|
||||||
mapping_file = 'single_table_mapping.json'
|
|
||||||
|
|
||||||
def setup():
|
|
||||||
t.setup()
|
|
||||||
|
|
||||||
def teardown():
|
|
||||||
t.teardown()
|
|
||||||
|
|
||||||
RELOFFSET = int(-1e17)
|
|
||||||
|
|
||||||
#######################################################################
|
|
||||||
def test_import():
|
|
||||||
"""Import succeeds"""
|
|
||||||
t.drop_schemas()
|
|
||||||
assert not t.table_exists('osm_all', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
t.imposm3_import(t.db_conf, './build/single_table.pbf', mapping_file)
|
|
||||||
assert t.table_exists('osm_all', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
|
|
||||||
def test_deploy():
|
|
||||||
"""Deploy succeeds"""
|
|
||||||
assert not t.table_exists('osm_all', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
t.imposm3_deploy(t.db_conf, mapping_file)
|
|
||||||
assert t.table_exists('osm_all', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
assert not t.table_exists('osm_all', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
|
|
||||||
#######################################################################
|
|
||||||
|
|
||||||
def test_non_mapped_node_is_missing():
|
|
||||||
"""Node without mapped tags is missing."""
|
|
||||||
t.assert_cached_node(10001, (10, 42))
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_all', 10001)
|
|
||||||
|
|
||||||
def test_mapped_node():
|
|
||||||
"""Node is stored with all tags."""
|
|
||||||
t.assert_cached_node(10002, (11, 42))
|
|
||||||
poi = t.query_row(t.db_conf, 'osm_all', 10002)
|
|
||||||
assert poi['tags'] == {'random': 'tag', 'but': 'mapped', 'poi': 'unicorn'}
|
|
||||||
|
|
||||||
|
|
||||||
def test_non_mapped_way_is_missing():
|
|
||||||
"""Way without mapped tags is missing."""
|
|
||||||
t.assert_cached_way(20101)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_all', 20101)
|
|
||||||
t.assert_cached_way(20102)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_all', 20102)
|
|
||||||
t.assert_cached_way(20103)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_all', 20103)
|
|
||||||
|
|
||||||
def test_mapped_way():
|
|
||||||
"""Way is stored with all tags."""
|
|
||||||
t.assert_cached_way(20201)
|
|
||||||
highway = t.query_row(t.db_conf, 'osm_all', -20201)
|
|
||||||
assert highway['tags'] == {'random': 'tag', 'highway': 'yes'}
|
|
||||||
|
|
||||||
def test_non_mapped_closed_way_is_missing():
|
|
||||||
"""Closed way without mapped tags is missing."""
|
|
||||||
t.assert_cached_way(20301)
|
|
||||||
assert not t.query_row(t.db_conf, 'osm_all', -20301)
|
|
||||||
|
|
||||||
def test_mapped_closed_way():
|
|
||||||
"""Closed way is stored with all tags."""
|
|
||||||
t.assert_cached_way(20401)
|
|
||||||
building = t.query_row(t.db_conf, 'osm_all', -20401)
|
|
||||||
assert building['tags'] == {'random': 'tag', 'building': 'yes'}
|
|
||||||
|
|
||||||
def test_mapped_closed_way_area_yes():
|
|
||||||
"""Closed way with area=yes is not stored as linestring."""
|
|
||||||
t.assert_cached_way(20501)
|
|
||||||
elem = t.query_row(t.db_conf, 'osm_all', -20501)
|
|
||||||
assert elem['geometry'].type == 'Polygon', elem['geometry'].type
|
|
||||||
assert elem['tags'] == {'random': 'tag', 'landuse': 'grass', 'highway': 'pedestrian', 'area': 'yes'}
|
|
||||||
|
|
||||||
def test_mapped_closed_way_area_no():
|
|
||||||
"""Closed way with area=no is not stored as polygon."""
|
|
||||||
t.assert_cached_way(20502)
|
|
||||||
elem = t.query_row(t.db_conf, 'osm_all', -20502)
|
|
||||||
assert elem['geometry'].type == 'LineString', elem['geometry'].type
|
|
||||||
assert elem['tags'] == {'random': 'tag', 'landuse': 'grass', 'highway': 'pedestrian', 'area': 'no'}
|
|
||||||
|
|
||||||
def test_mapped_closed_way_without_area():
|
|
||||||
"""Closed way without area is stored as mapped (linestring and polygon)."""
|
|
||||||
t.assert_cached_way(20601)
|
|
||||||
elems = t.query_row(t.db_conf, 'osm_all', -20601)
|
|
||||||
assert len(elems) == 2
|
|
||||||
elems.sort(key=lambda x: x['geometry'].type)
|
|
||||||
|
|
||||||
assert elems[0]['geometry'].type == 'LineString', elems[0]['geometry'].type
|
|
||||||
assert elems[0]['tags'] == {'random': 'tag', 'landuse': 'grass', 'highway': 'pedestrian'}
|
|
||||||
assert elems[1]['geometry'].type == 'Polygon', elems[1]['geometry'].type
|
|
||||||
assert elems[1]['tags'] == {'random': 'tag', 'landuse': 'grass', 'highway': 'pedestrian'}
|
|
||||||
|
|
||||||
def test_duplicate_ids_1():
|
|
||||||
"""Points/lines/polygons with same ID are inserted."""
|
|
||||||
node = t.query_row(t.db_conf, 'osm_all', 31101)
|
|
||||||
assert node['geometry'].type == 'Point', node['geometry'].type
|
|
||||||
assert node['tags'] == {'amenity': 'cafe'}
|
|
||||||
assert node['geometry'].distance(t.merc_point(80, 47)) < 1
|
|
||||||
|
|
||||||
ways = t.query_row(t.db_conf, 'osm_all', -31101)
|
|
||||||
ways.sort(key=lambda x: x['geometry'].type)
|
|
||||||
assert ways[0]['geometry'].type == 'LineString', ways[0]['geometry'].type
|
|
||||||
assert ways[0]['tags'] == {'landuse': 'park', 'highway': 'secondary'}
|
|
||||||
assert ways[1]['geometry'].type == 'Polygon', ways[1]['geometry'].type
|
|
||||||
assert ways[1]['tags'] == {'landuse': 'park', 'highway': 'secondary'}
|
|
||||||
|
|
||||||
rel = t.query_row(t.db_conf, 'osm_all', RELOFFSET-31101L)
|
|
||||||
assert rel['geometry'].type == 'Polygon', rel['geometry'].type
|
|
||||||
assert rel['tags'] == {'building': 'yes'}
|
|
||||||
|
|
||||||
|
|
||||||
#######################################################################
|
|
||||||
|
|
||||||
def test_update():
|
|
||||||
"""Diff import applies"""
|
|
||||||
t.imposm3_update(t.db_conf, './build/single_table.osc.gz', mapping_file)
|
|
||||||
|
|
||||||
#######################################################################
|
|
||||||
|
|
||||||
def test_duplicate_ids_2():
|
|
||||||
"""Node moved and ways/rels with same ID are still present."""
|
|
||||||
node = t.query_row(t.db_conf, 'osm_all', 31101)
|
|
||||||
assert node['geometry'].type == 'Point', node['geometry'].type
|
|
||||||
assert node['tags'] == {'amenity': 'cafe'}
|
|
||||||
assert node['geometry'].distance(t.merc_point(81, 47)) < 1
|
|
||||||
|
|
||||||
ways = t.query_row(t.db_conf, 'osm_all', -31101)
|
|
||||||
ways.sort(key=lambda x: x['geometry'].type)
|
|
||||||
|
|
||||||
assert ways[0]['geometry'].type == 'LineString', ways[0]['geometry'].type
|
|
||||||
assert ways[0]['tags'] == {'landuse': 'park', 'highway': 'secondary'}
|
|
||||||
assert ways[1]['geometry'].type == 'Polygon', ways[1]['geometry'].type
|
|
||||||
assert ways[1]['tags'] == {'landuse': 'park', 'highway': 'secondary'}
|
|
||||||
|
|
||||||
rel = t.query_row(t.db_conf, 'osm_all', RELOFFSET-31101L)
|
|
||||||
|
|
||||||
assert rel['geometry'].type == 'Polygon', rel['geometry'].type
|
|
||||||
assert rel['tags'] == {'building': 'yes'}
|
|
||||||
|
|
||||||
#######################################################################
|
|
||||||
def test_deploy_and_revert_deploy():
|
|
||||||
"""Revert deploy succeeds"""
|
|
||||||
assert not t.table_exists('osm_all', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
assert t.table_exists('osm_all', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
assert not t.table_exists('osm_all', schema=t.TEST_SCHEMA_BACKUP)
|
|
||||||
|
|
||||||
# import again to have a new import schema
|
|
||||||
t.imposm3_import(t.db_conf, './build/single_table.pbf', mapping_file)
|
|
||||||
assert t.table_exists('osm_all', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
|
|
||||||
t.imposm3_deploy(t.db_conf, mapping_file)
|
|
||||||
assert not t.table_exists('osm_all', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
assert t.table_exists('osm_all', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
assert t.table_exists('osm_all', schema=t.TEST_SCHEMA_BACKUP)
|
|
||||||
|
|
||||||
t.imposm3_revert_deploy(t.db_conf, mapping_file)
|
|
||||||
assert t.table_exists('osm_all', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
assert t.table_exists('osm_all', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
assert not t.table_exists('osm_all', schema=t.TEST_SCHEMA_BACKUP)
|
|
||||||
|
|
||||||
def test_remove_backup():
|
|
||||||
"""Remove backup succeeds"""
|
|
||||||
assert t.table_exists('osm_all', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
assert t.table_exists('osm_all', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
assert not t.table_exists('osm_all', schema=t.TEST_SCHEMA_BACKUP)
|
|
||||||
|
|
||||||
t.imposm3_deploy(t.db_conf, mapping_file)
|
|
||||||
|
|
||||||
assert not t.table_exists('osm_all', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
assert t.table_exists('osm_all', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
assert t.table_exists('osm_all', schema=t.TEST_SCHEMA_BACKUP)
|
|
||||||
|
|
||||||
t.imposm3_remove_backups(t.db_conf, mapping_file)
|
|
||||||
|
|
||||||
assert not t.table_exists('osm_all', schema=t.TEST_SCHEMA_IMPORT)
|
|
||||||
assert t.table_exists('osm_all', schema=t.TEST_SCHEMA_PRODUCTION)
|
|
||||||
assert not t.table_exists('osm_all', schema=t.TEST_SCHEMA_BACKUP)
|
|
||||||
|
|
|
@ -9,17 +9,19 @@ import (
|
||||||
"github.com/omniscale/imposm3/element"
|
"github.com/omniscale/imposm3/element"
|
||||||
"github.com/omniscale/imposm3/expire"
|
"github.com/omniscale/imposm3/expire"
|
||||||
geomp "github.com/omniscale/imposm3/geom"
|
geomp "github.com/omniscale/imposm3/geom"
|
||||||
"github.com/omniscale/imposm3/geom/geos"
|
geosp "github.com/omniscale/imposm3/geom/geos"
|
||||||
"github.com/omniscale/imposm3/mapping"
|
"github.com/omniscale/imposm3/mapping"
|
||||||
"github.com/omniscale/imposm3/stats"
|
"github.com/omniscale/imposm3/stats"
|
||||||
)
|
)
|
||||||
|
|
||||||
type RelationWriter struct {
|
type RelationWriter struct {
|
||||||
OsmElemWriter
|
OsmElemWriter
|
||||||
singleIdSpace bool
|
singleIdSpace bool
|
||||||
rel chan *element.Relation
|
rel chan *element.Relation
|
||||||
polygonMatcher mapping.RelWayMatcher
|
polygonMatcher mapping.RelWayMatcher
|
||||||
maxGap float64
|
relationMatcher mapping.RelationMatcher
|
||||||
|
relationMemberMatcher mapping.RelationMatcher
|
||||||
|
maxGap float64
|
||||||
}
|
}
|
||||||
|
|
||||||
func NewRelationWriter(
|
func NewRelationWriter(
|
||||||
|
@ -30,6 +32,8 @@ func NewRelationWriter(
|
||||||
inserter database.Inserter,
|
inserter database.Inserter,
|
||||||
progress *stats.Statistics,
|
progress *stats.Statistics,
|
||||||
matcher mapping.RelWayMatcher,
|
matcher mapping.RelWayMatcher,
|
||||||
|
relMatcher mapping.RelationMatcher,
|
||||||
|
relMemberMatcher mapping.RelationMatcher,
|
||||||
srid int,
|
srid int,
|
||||||
) *OsmElemWriter {
|
) *OsmElemWriter {
|
||||||
maxGap := 1e-1 // 0.1m
|
maxGap := 1e-1 // 0.1m
|
||||||
|
@ -45,10 +49,12 @@ func NewRelationWriter(
|
||||||
inserter: inserter,
|
inserter: inserter,
|
||||||
srid: srid,
|
srid: srid,
|
||||||
},
|
},
|
||||||
singleIdSpace: singleIdSpace,
|
singleIdSpace: singleIdSpace,
|
||||||
polygonMatcher: matcher,
|
polygonMatcher: matcher,
|
||||||
rel: rel,
|
relationMatcher: relMatcher,
|
||||||
maxGap: maxGap,
|
relationMemberMatcher: relMemberMatcher,
|
||||||
|
rel: rel,
|
||||||
|
maxGap: maxGap,
|
||||||
}
|
}
|
||||||
rw.OsmElemWriter.writer = &rw
|
rw.OsmElemWriter.writer = &rw
|
||||||
return &rw.OsmElemWriter
|
return &rw.OsmElemWriter
|
||||||
|
@ -62,7 +68,7 @@ func (rw *RelationWriter) relId(id int64) int64 {
|
||||||
}
|
}
|
||||||
|
|
||||||
func (rw *RelationWriter) loop() {
|
func (rw *RelationWriter) loop() {
|
||||||
geos := geos.NewGeos()
|
geos := geosp.NewGeos()
|
||||||
geos.SetHandleSrid(rw.srid)
|
geos.SetHandleSrid(rw.srid)
|
||||||
defer geos.Finish()
|
defer geos.Finish()
|
||||||
|
|
||||||
|
@ -74,9 +80,9 @@ NextRel:
|
||||||
if err != cache.NotFound {
|
if err != cache.NotFound {
|
||||||
log.Warn(err)
|
log.Warn(err)
|
||||||
}
|
}
|
||||||
continue NextRel
|
continue
|
||||||
}
|
}
|
||||||
for _, m := range r.Members {
|
for i, m := range r.Members {
|
||||||
if m.Way == nil {
|
if m.Way == nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
@ -88,65 +94,88 @@ NextRel:
|
||||||
continue NextRel
|
continue NextRel
|
||||||
}
|
}
|
||||||
rw.NodesToSrid(m.Way.Nodes)
|
rw.NodesToSrid(m.Way.Nodes)
|
||||||
|
r.Members[i].Elem = &m.Way.OSMElem
|
||||||
}
|
}
|
||||||
|
|
||||||
// BuildRelation updates r.Members but we need all of them
|
// handleRelation updates r.Members but we need all of them
|
||||||
// for the diffCache
|
// for the diffCache
|
||||||
allMembers := r.Members
|
allMembers := r.Members
|
||||||
|
|
||||||
// prepare relation first (build rings and compute actual
|
inserted := false
|
||||||
// relation tags)
|
|
||||||
prepedRel, err := geomp.PrepareRelation(r, rw.srid, rw.maxGap)
|
if handleRelationMembers(rw, r, geos) {
|
||||||
if err != nil {
|
inserted = true
|
||||||
if errl, ok := err.(ErrorLevel); !ok || errl.Level() > 0 {
|
}
|
||||||
log.Warn(err)
|
if handleRelation(rw, r, geos) {
|
||||||
}
|
inserted = true
|
||||||
continue NextRel
|
}
|
||||||
|
if handleMultiPolygon(rw, r, geos) {
|
||||||
|
inserted = true
|
||||||
}
|
}
|
||||||
|
|
||||||
// check for matches befor building the geometry
|
if inserted && rw.diffCache != nil {
|
||||||
matches := rw.polygonMatcher.MatchRelation(r)
|
rw.diffCache.Ways.AddFromMembers(r.Id, allMembers)
|
||||||
if len(matches) == 0 {
|
rw.diffCache.CoordsRel.AddFromMembers(r.Id, allMembers)
|
||||||
continue NextRel
|
for _, member := range allMembers {
|
||||||
}
|
if member.Way != nil {
|
||||||
|
rw.diffCache.Coords.AddFromWay(member.Way)
|
||||||
// build the multipolygon
|
|
||||||
geom, err := prepedRel.Build()
|
|
||||||
if err != nil {
|
|
||||||
if geom.Geom != nil {
|
|
||||||
geos.Destroy(geom.Geom)
|
|
||||||
}
|
|
||||||
if errl, ok := err.(ErrorLevel); !ok || errl.Level() > 0 {
|
|
||||||
log.Warn(err)
|
|
||||||
}
|
|
||||||
continue NextRel
|
|
||||||
}
|
|
||||||
|
|
||||||
if rw.limiter != nil {
|
|
||||||
start := time.Now()
|
|
||||||
parts, err := rw.limiter.Clip(geom.Geom)
|
|
||||||
if err != nil {
|
|
||||||
log.Warn(err)
|
|
||||||
continue NextRel
|
|
||||||
}
|
|
||||||
if duration := time.Now().Sub(start); duration > time.Minute {
|
|
||||||
log.Warnf("clipping relation %d to -limitto took %s", r.Id, duration)
|
|
||||||
}
|
|
||||||
for _, g := range parts {
|
|
||||||
rel := element.Relation(*r)
|
|
||||||
rel.Id = rw.relId(r.Id)
|
|
||||||
geom = geomp.Geometry{Geom: g, Wkb: geos.AsEwkbHex(g)}
|
|
||||||
err := rw.inserter.InsertPolygon(rel.OSMElem, geom, matches)
|
|
||||||
if err != nil {
|
|
||||||
if errl, ok := err.(ErrorLevel); !ok || errl.Level() > 0 {
|
|
||||||
log.Warn(err)
|
|
||||||
}
|
|
||||||
continue
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} else {
|
}
|
||||||
|
if inserted && rw.expireor != nil {
|
||||||
|
for _, m := range allMembers {
|
||||||
|
if m.Way != nil {
|
||||||
|
expire.ExpireNodes(rw.expireor, m.Way.Nodes)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rw.wg.Done()
|
||||||
|
}
|
||||||
|
|
||||||
|
func handleMultiPolygon(rw *RelationWriter, r *element.Relation, geos *geosp.Geos) bool {
|
||||||
|
// prepare relation first (build rings and compute actual
|
||||||
|
// relation tags)
|
||||||
|
prepedRel, err := geomp.PrepareRelation(r, rw.srid, rw.maxGap)
|
||||||
|
if err != nil {
|
||||||
|
if errl, ok := err.(ErrorLevel); !ok || errl.Level() > 0 {
|
||||||
|
log.Warn(err)
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// check for matches befor building the geometry
|
||||||
|
matches := rw.polygonMatcher.MatchRelation(r)
|
||||||
|
if matches == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// build the multipolygon
|
||||||
|
geom, err := prepedRel.Build()
|
||||||
|
if geom.Geom != nil {
|
||||||
|
defer geos.Destroy(geom.Geom)
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
if errl, ok := err.(ErrorLevel); !ok || errl.Level() > 0 {
|
||||||
|
log.Warn(err)
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
if rw.limiter != nil {
|
||||||
|
start := time.Now()
|
||||||
|
parts, err := rw.limiter.Clip(geom.Geom)
|
||||||
|
if err != nil {
|
||||||
|
log.Warn(err)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if duration := time.Now().Sub(start); duration > time.Minute {
|
||||||
|
log.Warnf("clipping relation %d to -limitto took %s", r.Id, duration)
|
||||||
|
}
|
||||||
|
for _, g := range parts {
|
||||||
rel := element.Relation(*r)
|
rel := element.Relation(*r)
|
||||||
rel.Id = rw.relId(r.Id)
|
rel.Id = rw.relId(r.Id)
|
||||||
|
geom = geomp.Geometry{Geom: g, Wkb: geos.AsEwkbHex(g)}
|
||||||
err := rw.inserter.InsertPolygon(rel.OSMElem, geom, matches)
|
err := rw.inserter.InsertPolygon(rel.OSMElem, geom, matches)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if errl, ok := err.(ErrorLevel); !ok || errl.Level() > 0 {
|
if errl, ok := err.(ErrorLevel); !ok || errl.Level() > 0 {
|
||||||
|
@ -155,29 +184,103 @@ NextRel:
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
} else {
|
||||||
for _, m := range mapping.SelectRelationPolygons(rw.polygonMatcher, r) {
|
rel := element.Relation(*r)
|
||||||
err = rw.osmCache.InsertedWays.PutWay(m.Way)
|
rel.Id = rw.relId(r.Id)
|
||||||
if err != nil {
|
err := rw.inserter.InsertPolygon(rel.OSMElem, geom, matches)
|
||||||
|
if err != nil {
|
||||||
|
if errl, ok := err.(ErrorLevel); !ok || errl.Level() > 0 {
|
||||||
log.Warn(err)
|
log.Warn(err)
|
||||||
}
|
}
|
||||||
|
return false
|
||||||
}
|
}
|
||||||
if rw.diffCache != nil {
|
|
||||||
rw.diffCache.Ways.AddFromMembers(r.Id, allMembers)
|
|
||||||
for _, member := range allMembers {
|
|
||||||
if member.Way != nil {
|
|
||||||
rw.diffCache.Coords.AddFromWay(member.Way)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if rw.expireor != nil {
|
|
||||||
for _, m := range allMembers {
|
|
||||||
if m.Way != nil {
|
|
||||||
expire.ExpireNodes(rw.expireor, m.Way.Nodes)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
geos.Destroy(geom.Geom)
|
|
||||||
}
|
}
|
||||||
rw.wg.Done()
|
|
||||||
|
for _, m := range mapping.SelectRelationPolygons(rw.polygonMatcher, r) {
|
||||||
|
err = rw.osmCache.InsertedWays.PutWay(m.Way)
|
||||||
|
if err != nil {
|
||||||
|
log.Warn(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func handleRelation(rw *RelationWriter, r *element.Relation, geos *geosp.Geos) bool {
|
||||||
|
relMatches := rw.relationMatcher.MatchRelation(r)
|
||||||
|
if relMatches == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
rel := element.Relation(*r)
|
||||||
|
rel.Id = rw.relId(r.Id)
|
||||||
|
rw.inserter.InsertPolygon(rel.OSMElem, geomp.Geometry{}, relMatches)
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func handleRelationMembers(rw *RelationWriter, r *element.Relation, geos *geosp.Geos) bool {
|
||||||
|
relMemberMatches := rw.relationMemberMatcher.MatchRelation(r)
|
||||||
|
if relMemberMatches == nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for i, m := range r.Members {
|
||||||
|
if m.Type == element.RELATION {
|
||||||
|
mrel, err := rw.osmCache.Relations.GetRelation(m.Id)
|
||||||
|
if err != nil {
|
||||||
|
if err == cache.NotFound {
|
||||||
|
log.Warn(err)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
r.Members[i].Elem = &mrel.OSMElem
|
||||||
|
} else if m.Type == element.NODE {
|
||||||
|
nd, err := rw.osmCache.Nodes.GetNode(m.Id)
|
||||||
|
if err != nil {
|
||||||
|
if err == cache.NotFound {
|
||||||
|
nd, err = rw.osmCache.Coords.GetCoord(m.Id)
|
||||||
|
if err != nil {
|
||||||
|
if err != cache.NotFound {
|
||||||
|
log.Warn(err)
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
log.Warn(err)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rw.NodeToSrid(nd)
|
||||||
|
r.Members[i].Node = nd
|
||||||
|
r.Members[i].Elem = &nd.OSMElem
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, m := range r.Members {
|
||||||
|
var g *geosp.Geom
|
||||||
|
var err error
|
||||||
|
if m.Node != nil {
|
||||||
|
g, err = geomp.Point(geos, *m.Node)
|
||||||
|
} else if m.Way != nil {
|
||||||
|
g, err = geomp.LineString(geos, m.Way.Nodes)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
log.Warn(err)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
var gelem geomp.Geometry
|
||||||
|
if g == nil {
|
||||||
|
g = geos.FromWkt("POLYGON EMPTY")
|
||||||
|
gelem = geomp.Geometry{Geom: g, Wkb: geos.AsEwkbHex(g)}
|
||||||
|
} else {
|
||||||
|
gelem, err = geomp.AsGeomElement(geos, g)
|
||||||
|
if err != nil {
|
||||||
|
log.Warn(err)
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rel := element.Relation(*r)
|
||||||
|
rel.Id = rw.relId(r.Id)
|
||||||
|
rw.inserter.InsertRelationMember(rel, m, gelem, relMemberMatches)
|
||||||
|
}
|
||||||
|
return true
|
||||||
}
|
}
|
||||||
|
|
|
@ -129,6 +129,9 @@ func (ww *WayWriter) buildAndInsert(g *geos.Geos, w *element.Way, matches []mapp
|
||||||
|
|
||||||
if isPolygon {
|
if isPolygon {
|
||||||
geosgeom, err = geomp.Polygon(g, way.Nodes)
|
geosgeom, err = geomp.Polygon(g, way.Nodes)
|
||||||
|
if err == nil {
|
||||||
|
geosgeom, err = g.MakeValid(geosgeom)
|
||||||
|
}
|
||||||
} else {
|
} else {
|
||||||
geosgeom, err = geomp.LineString(g, way.Nodes)
|
geosgeom, err = geomp.LineString(g, way.Nodes)
|
||||||
}
|
}
|
||||||
|
|
Loading…
Reference in New Issue