Franchu’s Lair

Last year I attended GEOSTAT 2009, and it was a great experience! This is the reason why I think that if you are into geostatistics you may want to take a look at the call for participation for this years edition that will take place in Plasencia, Spain.

The original call for participation read like this:

GEOSTAT focuses on important aspects of statistical analysis of spatial and spatio-temporal data using open source / free GIS tools: R, SAGA GIS, GRASS GIS, Quantum GIS, GDAL, Google Earth and similar. The course participants learn how to move data back and forth between different environments, how to produce scripts and automate analysis. We welcome also R beginners and users needing refresh courses in
programming. This year, we would also like to introduce/promote topics such as: web-based computing, WPS client-server environments, 3D and 4D geostatistics, combining R+SAGA/GRASS. This is a 5-day course with two parallel sessions, which means that there will be total 7 full-day blocks (three days with parallel sessions) of lectures; the last day of the summer school participants can present their research problems and ask for feedback from the whole summer school. For more info see:

http://geostat2010.info/Info

In order to get some more information, I ran a short Q&A with Tomislav Hengl, the guy who started all this GEOSTAT thing and keeps organising them

Why are you doing GEOSTAT 2010?

Tomislav Hengl: The first such event I have organized was in 2004 in Zagreb, Croatia. I just came back from ITC, Netherlands (where I defended my PhD in pedometrics) and wanted to transfer and share some of my knowledge/experiences locally to colleagues in Croatia, who obviously had limited resources to follow international activities. In the meanwhile the event became international, and is now mainly dedicated to the open source spatial data analysis tools, primarily R and open source GIS (SAGA, GRASS). So my original motive to run it is the knowledge-sharing (’spread the word’). I also see it as a good investment – young researchers and people working on small projects get to see the benefits of using open source tools. They become a part of community, so that there is a bigger momentum to do even more ambitious projects. Sounds a bit as Jehovah’s Witnesses, but open source academic software is definitively a noble mission.

Are you going to do another edition of GEOSTAT in 2011?

Tomislav Hengl: Of course. As long as there is interest, there will be GEOSTAT (by looking at the number of registrations we get every year – we should do it at least 3 times in 2011!).

So… why should I try to go to GEOSTAT this year? What makes it special this time?

Tomislav Hengl: The GEOSTAT 2010 is special because of three things:

• this is the first time where both main developers of GRASS GIS, R spatial tools and SAGA GIS will come together (we have 7 lecturers in total!)
• this year we will run parallel training blocks, so that participants will have more flexibility to select topics that interest them the most
• Spanish beer feels great when you listen for a whole day about R commands on 35 degrees C!

I already have your book, is there something new for me at GEOSTAT?

Tomislav Hengl: But I bet you that you have many questions on things in the book that the book cannot answer. No seriously, every year the program is at least 30% new. This is simply because R+OSGeo tools evolve at increasing paste. If you really want to be on the edge, you should come and listen to Roger Bivand, Edzer Pebesma, Gerard Heuvelink, Markus Metz, Victor Olaya…

I don’t have publications, or I’m not a PhD student, but I really love geospatial… what are my chances of attending?

Tomislav Hengl: They can still be high. The selection criteria for this year are:

• time of application
• distance to the venue (more distant applications have an advantage – for simple reason of solidarity with people that have to invest a lot to reach Spain)
• contributions to the open source activities (R-sig-geo traffic, number of packages and plugins developed)
• publication record/academic level

So you see – even if you were a MSc student, you could have got in if you have applied on time and made some contributions to the community.

Is GEOSTAT about learning how to use tools or will I also get the theory behind it?

Tomislav Hengl: We really let the participants decide about this (democratically). About 40% of the programme is flexible, so lecturers can decide whether to take some steps back and re-discuss theoretical grounds (in most cases people have such grounds already, so the focus in typically more on mastering the software). My experience is that people like to learn things that are critically difficult to understand without getting detailed instructions (tips-n-tricks) from people who developed the tools (or theory) originally. In most cases this leads to understanding why was some algorithm designed in such way, what are its bottle-necks and how to go around them, and how to interpret results of analysis for various case studies. At the end, I do not think that you can completely separate between the tool and theory: you need tools (R+GIS) to be able to implement some theory, and you need theory that can fit your applications, so these are actually closely connected.

Why is GEOSTAT oriented to open source tools? Can’t I get the same results with commercial software?

Tomislav Hengl: This is my most important point that I always make when asked about my motives to use R (some think we do it just because it is for free). I use R primarily because of the quality and diversity of algorithms. Take for example geostatistics. If you know how to use R+gstat/geoR you can do so many things – fit non-linear models, plot variograms with confidence limits, plot space-time variograms, predict or generate simulations using regression-kriging etc etc. Try doing this with ArcGIS. Did you know that the main creator of Geostatistical analyst for ESRI has been trying to bring R functionality to ArcGIS for years? And I do not see that vice-versa is happening. There is also the issue of confidence and transparency – R is open source so you can see and track exactly what goes on. With commercial software (which is usually close code) you have to be a blind believer. I am definitively not a one.

I don’t know how to use R/SAGA/… nor how to program, do you think I can still benefit from GEOSTAT?

Tomislav Hengl: I personally have full understanding for the coding beginners, so they are welcome. We actually run every year a small block for beginners called “baby-steps-in-R” (or how to start understanding R in 2 hours). The participants need to have some motive, talent and enthusiasm to begin with, we will take care of helping them master the code. Now about the benefits of the whole thing… I am really not the right person to say. This is more a question for you to comment, right?

Indeed, the baby-steps-in-R are really useful, and even if during GEOSTAT you are challenged by the lack of skills in R you always learn a lot of things. Last year when I attended GEOSTAT I had lots of troubles with R, but everybody was eager to help you understand the big picture. Still, if you want to get the most out of it… prepare before going there!

Last year I ran a short interview with Roger Bivand, one of the lecturers that will be again at GEOSTAT 2010.

The deadline for application is the 15th February 2010, so you better rush if you want to make it on time. If you are not accepted this year, keep an eye on Tomislav’s website for future editions of GEOSTAT as it is something worth attending.

During GEOSTAT2009 earlier this year, I had the pleasure to meet Professor Roger Bivand, a british geographer teaching in a norwegian economics schools who was kind enough to spend a week in Croatia sharing his knowledge on geostatistics with us.

He is one of the authors of the sp package for R and he has a very good overview of the state-of-the-art on geostatistics.

The last day of the course, I proposed him to run a short interview on my blog and he accepted. His answers came very fast to my mailbox, but it took me way too long to sit down and write them in this post.

I would like to thank Roger for his kind answers, and I hope you will enjoy reading them as much as I did.

As one of the main developers of the sp package , what was your motivation to do it?

It’s mostly described in a talk I gave at DSC’03 in Vienna although the actual sketch was made on an excursion at the StatGIS meeting later the same year, at which Edzer Pebesma and Virgilio Gómez Rubio were also present, and where Albrecht Gebhardt was a local organiser.

The idea is not so much to provide a single standard representation, but to be open to interfacing with other representations, so that exchanging data may be made easier.

Has SpatialData (and their related structures) become the standard to represent spatial information in R? If any, which are the alternatives?

The large number of packages that depend on or suggest sp indicates that others find the representations useful. Other representations include those in the maps, PBSmapping, and spatstat packages among others – most are listed on the CRAN “Spatial” Task View.

Is it possible to contribute to the sp package or it is a piece of work that is complete and just maintained by a selected few?

The package source is (still) hosted on the r-spatial project on sourceforge. Contributions are welcome, but their incorporation is the responsibility of the authors, so it may be suggested that a separate package is more appropriate. I guess “select few” is not inappropriate, but that’s just the way things have happened.

What would you like to see implemented in the sp package that is still not there?

After four implementations of rings and polygons as something like a shapefile, it might be worth looking at an OGC Simple Features representation instead, otherwise nothing much on the TODO list.

Lately there has been a lot of talking about R (it got even an article on the New York Times) Have you noticed an increase on the interest on the sp package and the spatial capabilities of R?

There is a good deal of interest, witnessed by the number of subscribers on R-SIG-GEO, and offlist email traffic. But interest in itself isn’t interesting, it’s the quality of the new ideas and good bug reports that keeps the momentum up.

As the person who runs the R-SIG-GEO mailing list, how would you describe the community there? What are your plans (if any) for the mailing list?

There are an increasing number of “helpers” – subscribers who both start new threads and reply to questions in existing threads – it would be nice to see that trend continue and strengthen. It seems important to help subscribers to express themselves clearly – some problems are seen rather differently in other disciplines. I don’t have plans beyond trying to answer well-framed questions as they are posted, something which can take a good chunk of my day. I can feel guilty if someone has to wait, or doesn’t get an answer, but know that this isn’t a sustainable model, others can often answer better than I could.

As a geographer working in an economics school, you have a solid background on geography and statistics. What would be the baby steps that you would recommend to people lacking such a background to get initiated in the field of geostatistics up to the level in which they can follow the ASDAR book?

This is hard to answer, as people vary a lot. Some like to study a book or books – many are indicated in the ASDAR book, at different levels. Others are driven by a research question, others again learn through examples. Probably the only shared characteristic would be that these are things you can’t really get unmotivated people to grasp, but if someone is motivated, they’ll find their way in somehow. And enjoy it, after all, it is supposed to be fun, isn’t it?

Geostatistics is a relatively recent field and some of the calculations have been only possible in the last years thanks to the development of cheap computing power. Supposing that computing power and memory are not an issue… what do you think would be the next developments in the geostatistics world? (4D interpolation/prediction? local geostatistics?)

Hard to say, but probably more model-based geostatistics and conditional simulation (making the best use of limited data). Beyond that, I think that before 4D or even 3D, we should expect more movement on geostatistics for non-continuous data (count data and presence/absence data), something like GLMM models, and most likely hierarchical models capable of doing something sensible with zero inflation, which dogs many research problems. In fact, getting statisticians involved in helping design data collection is more likely to yield better results than more compute power.

One of the biggest challenges we need to resolve is related to information visualisation. The outputs of the calculations need to be easily understandable by others. Do you think that R is the right place to develop new visualisations or would it be better to export/communicate the data to other applications in which visual designers feel more comportable?

If we are thinking about analytical and exploratory visualisation, then yes, R is an appropriate place, not least because a lot of the research on statistical graphics and data visualization is done in R. Dynamic linked graphics with zooming are not easy, and will take time to do in a cross-platform way, so maybe there, other software environments may be prefered. The current graphics model in R is under constant development, with the newer achievement including anti-aliassing and proper use of the alpha channel where possible. This is also linked to attempts to provide proper graphics support for UTF-8, which is harder than one might think. A lot is going on at the moment in these areas, but time will tell which innovations succeed and diffuse.

Last month I had the chance to attend GEOSTAT2009, a summer school on spatio-temporal data analysis with R + SAGA + Google Earth that was held at MedILS in Split, Croatia.

The summer school was a great opportunity to get some hands on experience with R and SAGA, as learning them on your own can be a little bit tricky. We were also very lucky to have three teachers (Roger Bivand, Olaf Conrad and Tomislav Hengl) that have a lot of experience with those pieces of software and the field of geostatistics.

But one of the things I liked most of the course is that there was a wide variety of fields represented in the course as we had biologists, geologists, geographers, meteorologists, engineers and software developers! (I think I am still missing some fields…) It was an eye opening experience for me, as the same techniques can be applied to all those sciences and just the interpretation of the outputs differ to accommodate the processes involved in each discipline.

Given my experience in software development, I was able to grasp very quickly how to write the code to implement the steps needed to solve a problem. Nevertheless, my sub-par knowledge of geostatistics was the handicap I had to fight with during all the course. That is why I really enjoyed when we were doing the practical exercises and we were discussing with other participants. They always had the theoretical background to explain what was needed to do, and we were able to implement it faster than them. Symbiosis at its best

All this took place in a wonderful place, a former Tito’s residence (Villa Dalmatia), where one of the buildings of the complex has been reconverted into a life sciences research institute. We had our own private beach, basketball court, tennis table, and a great food catering with lots of variety. In such a wonderful environment it is difficult not to have fun and forge new friendships while learning really advanced stuff.

For the weekend, we went to Brac, an island that is in front of Split and we had a great day walking around to Pustinja Blaca (an old monastery in the mountains), and swimming in Bol.

You can see some of the photos I took during that week:

In the following days, work permitting, I will post some interesting interviews and ideas I got during the course.