Nov 14 2010

UCOSP — Converting Well Known Binary to KML">UCOSP — Converting Well Known Binary to KML

Hey guys!
So, last week I set out some goals for myself to get com­pleted this past week. I’ll re-post them here:

Have the arti­cle from Andrew read so I can be brought up to speed on some of the OGR ter­mi­nol­ogy.
Have OGR built with Opti­miza­tion on.
Have a sam­ple piece of C code going that demon­strates KML.
Hope­fully have the details from Chuck on what the changes are from list­ing the SQL func­tions. If this pans out, try to get a stub func­tion going that out­puts “Hello KML!”

I am proud to say I’ve achieved my goals! First thing that I did was to get the build for OGR going, so I will dis­cuss that first.

OGR

OGR is an open source library OGR is actu­ally a sub­set of the GDAL pack­age, so in order to build GDAL, it meant hav­ing to build OGR. Here is a link to the GDAL source:

wget http://download.osgeo.org/gdal/gdal172.zip

Andrew wanted to know how large OGR would be once opti­miza­tion was turned on, and so I set out to find out this info. I didn’t exactly know what it meant for opti­miza­tion to be on — but I learned that it meant that it was a build of OGR that would be pro­duc­tion ready, that min­i­mized the amount of debug data. After doing a build of OGR, it was deemed an accept­able sized library to include in Ingres! Here were the steps I used to build OGR:

export CFLAGS=-O2
export CXXFLAGS=-O2

And the configure:

./configure –prefix=/usr/local \
> –with-threads \
> –with-ogr \
> –with-geos \
> –without-libtool \
> –with-libz=internal \
> –with-libtiff=internal \
> –with-geotiff=internal \
> –without-gif \
> –without-pg \
> –without-grass \
> –without-libgrass \
> –without-cfitsio \
> –without-pcraster \
> –without-netcdf \
> –without-png \
> –with-jpeg=internal \
> –without-gif \
> –without-ogdi \
> –without-fme \
> –without-hdf4 \
> –without-hdf5 \
> –without-jasper \
> –without-ecw \
> –without-kakadu \
> –without-mrsid \
> –without-jp2mrsid \
> –without-bsb \
> –without-grib \
> –without-mysql \
> –without-ingres \
> –without-xerces \
> –without-expat \
> –without-odbc \
> –without-curl \
> –without-sqlite3 \
> –without-dwgdirect \
> –without-idb \
> –without-sde \
> –without-perl \
> –without-php \
> –without-ruby \
> –without-python \

Once I was able to com­pile, build, and install it, I was able to fol­low some of the sam­ple API stuff in OGR.

KML

The ques­tions sur­round­ing KML have been inter­est­ing, its been on and off as to which library to use to con­vert well known binary (WKB) into KML. The orig­i­nal idea was to use OGR, but then when talk­ing to the OGR com­mu­nity, they said that it may not be thread safe.. We started doing research into what else we could use, maybe just directly using libkml. If we were to go this route then we would have to cre­ate the algo­rithm that con­verts WKB into some­thing that we can enter into the libkml library. We went back to the OGR route, and with the help of Frank Warmer­dam, we were able to get WKB con­verted into KML! My biggest chal­lenge dur­ing this process was learn­ing how to com­pile and build exe­cuta­bles in C, using dynamic libraries, but after learn­ing, it was really quite sim­ple. I just haven’t done very much C :)

Here is the code that con­verted WKB into KML incase any­one wants to use this themselves:

#include “ogr_api.h”

int main()
{
OGR­Err eErr;
OGR­Ge­om­e­tryH hGeom = NULL;
char *kml = NULL;
unsigned char *wkb_buffer = “\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x14@\x00\x00\x00\x00\x00\x007@”;

eErr = OGR_G_CreateFromWkb( wkb_buffer, NULL, &hGeom, –1 );
if( eErr == OGRERR_NONE )
{
kml = OGR_G_ExportToKML( hGeom, “???I do not know what this is” );
OGR_G_DestroyGeometry( hGeom );
printf( “kml = %s\n”, kml );
//CPLFree( kml );
}
else if( eErr == OGRERR_NOT_ENOUGH_DATA )
{
printf(“error1”);
}
else if( eErr == OGRERR_UNSUPPORTED_GEOMETRY_TYPE )
{
printf(“error2”);
}
else if( eErr == OGRERR_CORRUPT_DATA )
{
printf(“error3”);
};

}

The next step now is to imple­ment the asKML func­tion within the Ingres source code! Right now one of the devel­op­ers Chuck is still work­ing on reviv­ing the instruc­tions to do this, but once he is fin­ished, I will be good to go!

Shawn


Nov 14 2010

Ingres Geospatial Formatting Functions Project Plan

Hey all!

Here is a link to my orig­i­nal post:

http://www.fosslc.org/drupal/content/ingres-geospatial-formatting-functions-project-plan

So, some updates to talk about since my last blog entry (this will be hap­pen­ing every Sun­day). Andrew Ross gave us an out­line, so I think I will run with that for this post!

Key Require­ments


Some of these have already been listed in my first blog post, but I will just reit­er­ate them:
1. Cre­ate cus­tom SQL func­tions (mine being asKML, other mem­bers writ­ing asGML and asGeoJSON).
2. Inte­grate OGR into Ingres.

Approach

1. Learn how to cre­ate cus­tom SQL func­tions, and ini­tially make asKML out­put “Hello KML” each time you use it, just so I know that I have cre­ated an SQL func­tion cor­rectly. I will be doing this with guidence from the fol­low­ing arti­cle in the documentation:
http://community.ingres.com/wiki/Implementing_New_SQL_Function
I am also plan­ning on look­ing through some of Evas revi­sions to see some of the diffs on what she did to add func­tions. Although one of the full time Ingres devel­op­ers changed the way func­tions are listed, so at the moment the geospa­tial branch is bro­ken. Once this is all resolved, we will be briefed on what we need to do to make these func­tions work.
2. Last time this #2 men­tioned that we would take PIECES of OGR to do asKML, asGML, and asGeo­J­SON. Now the new plan dis­cussed with Andrew is to include the entire OGR/GDAL library as part of Ingres. The rea­son for this is because it might get complicated/messy to do a copy and paste of the OGR code. Also, there are many things in the library that could be use­ful, such as other as*** func­tions, as well as use­ful for one of the main devel­op­ers Chuck for his work on raster image support.

Ques­tions


1. It was men­tioned in an email that we were going to get a link to an arti­cle you cre­ated that will help us ramp up on Geospa­tial terms. This isn’t really a ques­tion per say, but I have had a bit of con­fu­sion when it comes to the ter­mi­nol­ogy, and instead of ask­ing specifics I’d like to have a read at that article.
2. Andrew, you men­tioned at some point that you would like to check GDAL for thread safety. How would we go about doing this?
3. I would like to get a sta­tus update on Chucks progress for what changed for the way func­tions are listed, and also if the doc­u­men­ta­tion will be updated on how to get func­tions work­ing. The instruc­tions are cur­rently VERY pre­cise, and I’m scared that any slight change will have me very con­fused when try­ing to make a func­tion work.

Test­ing Plans


I haven’t dived too much into my test­ing plans for this yet. What I was think­ing was work­ing with Jor­dan on writ­ing some test plans on this, as he is the stu­dent on this project work­ing on test­ing. But this is com­pletely up in the air for me, and Andrew, I’d like to hear your feed­back on the plan for this :)

Week by Week Plan

Oct 17th — 24th
  • Have the arti­cle from Andrew read so I can be brought up to speed on some of the OGR terminology.
  • Have OGR built with Opti­miza­tion on.
  • Have a sam­ple piece of C code going that demon­strates KML.
  • Hope­fully have the details from Chuck on what the changes are from list­ing the SQL func­tions. If this pans out, try to get a stub func­tion going that out­puts “Hello KML!”
Oct 24th — 31th
  • Last week to have a Stub func­tion imple­mented that out­puts “Hello KML!”.
  • Dive more into the SQL func­tion­al­ity. See how I can take the out­put, can actu­ally apply func­tions to it. Right now, I have no idea where asText and asBi­nary actu­ally get the data from, and that is a con­cern of mine. Poten­tially will have to find the right peo­ple to ask this question.
Oct 31st — Nov. 7th
  • Once I fig­ure out the SQL stuff, see if I can inte­grate the func­tions into the asKML func­tion I have from OGR.
  • This will require some ramp up of learn­ing C libraries, as I have not really coded any­thing in C. Once it is built and installed, learn how to call the C API through Ingres source code.
Nov 7th — 14th
  • Last week to get asKML imple­mented as a func­tion, what­ever this may entail at this point.
  • Talk to Alex Tro­fast to teach us how to get OGR into the build process.
Nov 14th — 21st
  • Start writ­ing up test plan doc­u­men­ta­tion and implementation
  • Bug fix­ing
  • Ensure build process includes OGR
Nov 21st — 28th
  • Last week of doc­u­men­ta­tion, bug fixes, ensur­ing cor­rect­ness and quality.