The original call for participation read like this:

GEOSTAT focuses on important aspects of statistical analysis of spatial and spatio-temporal data using open source / free GIS tools: R, SAGA GIS, GRASS GIS, Quantum GIS, GDAL, Google Earth and similar. The course participants learn how to move data back and forth between different environments, how to produce scripts and automate analysis. We welcome also R beginners and users needing refresh courses in
programming. This year, we would also like to introduce/promote topics such as: web-based computing, WPS client-server environments, 3D and 4D geostatistics, combining R+SAGA/GRASS. This is a 5-day course with two parallel sessions, which means that there will be total 7 full-day blocks (three days with parallel sessions) of lectures; the last day of the summer school participants can present their research problems and ask for feedback from the whole summer school. For more info see:

http://geostat2010.info/Info

In order to get some more information, I ran a short Q&A with Tomislav Hengl, the guy who started all this GEOSTAT thing and keeps organising them

Why are you doing GEOSTAT 2010?

Tomislav Hengl: The first such event I have organized was in 2004 in Zagreb, Croatia. I just came back from ITC, Netherlands (where I defended my PhD in pedometrics) and wanted to transfer and share some of my knowledge/experiences locally to colleagues in Croatia, who obviously had limited resources to follow international activities. In the meanwhile the event became international, and is now mainly dedicated to the open source spatial data analysis tools, primarily R and open source GIS (SAGA, GRASS). So my original motive to run it is the knowledge-sharing (’spread the word’). I also see it as a good investment – young researchers and people working on small projects get to see the benefits of using open source tools. They become a part of community, so that there is a bigger momentum to do even more ambitious projects. Sounds a bit as Jehovah’s Witnesses, but open source academic software is definitively a noble mission.

Are you going to do another edition of GEOSTAT in 2011?

Tomislav Hengl: Of course. As long as there is interest, there will be GEOSTAT (by looking at the number of registrations we get every year – we should do it at least 3 times in 2011!).

So… why should I try to go to GEOSTAT this year? What makes it special this time?

Tomislav Hengl: The GEOSTAT 2010 is special because of three things:

• this is the first time where both main developers of GRASS GIS, R spatial tools and SAGA GIS will come together (we have 7 lecturers in total!)
• this year we will run parallel training blocks, so that participants will have more flexibility to select topics that interest them the most
• Spanish beer feels great when you listen for a whole day about R commands on 35 degrees C!

I already have your book, is there something new for me at GEOSTAT?

Tomislav Hengl: But I bet you that you have many questions on things in the book that the book cannot answer. No seriously, every year the program is at least 30% new. This is simply because R+OSGeo tools evolve at increasing paste. If you really want to be on the edge, you should come and listen to Roger Bivand, Edzer Pebesma, Gerard Heuvelink, Markus Metz, Victor Olaya…

I don’t have publications, or I’m not a PhD student, but I really love geospatial… what are my chances of attending?

Tomislav Hengl: They can still be high. The selection criteria for this year are:

• time of application
• distance to the venue (more distant applications have an advantage – for simple reason of solidarity with people that have to invest a lot to reach Spain)
• contributions to the open source activities (R-sig-geo traffic, number of packages and plugins developed)
• publication record/academic level

So you see – even if you were a MSc student, you could have got in if you have applied on time and made some contributions to the community.

Is GEOSTAT about learning how to use tools or will I also get the theory behind it?

Tomislav Hengl: We really let the participants decide about this (democratically). About 40% of the programme is flexible, so lecturers can decide whether to take some steps back and re-discuss theoretical grounds (in most cases people have such grounds already, so the focus in typically more on mastering the software). My experience is that people like to learn things that are critically difficult to understand without getting detailed instructions (tips-n-tricks) from people who developed the tools (or theory) originally. In most cases this leads to understanding why was some algorithm designed in such way, what are its bottle-necks and how to go around them, and how to interpret results of analysis for various case studies. At the end, I do not think that you can completely separate between the tool and theory: you need tools (R+GIS) to be able to implement some theory, and you need theory that can fit your applications, so these are actually closely connected.

Why is GEOSTAT oriented to open source tools? Can’t I get the same results with commercial software?

Tomislav Hengl: This is my most important point that I always make when asked about my motives to use R (some think we do it just because it is for free). I use R primarily because of the quality and diversity of algorithms. Take for example geostatistics. If you know how to use R+gstat/geoR you can do so many things – fit non-linear models, plot variograms with confidence limits, plot space-time variograms, predict or generate simulations using regression-kriging etc etc. Try doing this with ArcGIS. Did you know that the main creator of Geostatistical analyst for ESRI has been trying to bring R functionality to ArcGIS for years? And I do not see that vice-versa is happening. There is also the issue of confidence and transparency – R is open source so you can see and track exactly what goes on. With commercial software (which is usually close code) you have to be a blind believer. I am definitively not a one.

I don’t know how to use R/SAGA/… nor how to program, do you think I can still benefit from GEOSTAT?

Tomislav Hengl: I personally have full understanding for the coding beginners, so they are welcome. We actually run every year a small block for beginners called “baby-steps-in-R” (or how to start understanding R in 2 hours). The participants need to have some motive, talent and enthusiasm to begin with, we will take care of helping them master the code. Now about the benefits of the whole thing… I am really not the right person to say. This is more a question for you to comment, right?

Indeed, the baby-steps-in-R are really useful, and even if during GEOSTAT you are challenged by the lack of skills in R you always learn a lot of things. Last year when I attended GEOSTAT I had lots of troubles with R, but everybody was eager to help you understand the big picture. Still, if you want to get the most out of it… prepare before going there!

Last year I ran a short interview with Roger Bivand, one of the lecturers that will be again at GEOSTAT 2010.

The deadline for application is the 15th February 2010, so you better rush if you want to make it on time. If you are not accepted this year, keep an eye on Tomislav’s website for future editions of GEOSTAT as it is something worth attending.

Related posts:

1. GEOSTAT2009 Last month I had the chance to attend GEOSTAT2009, a...
2. Conversations with Professor Roger Bivand During GEOSTAT2009 earlier this year, I had the pleasure...

Related posts brought to you by Yet Another Related Posts Plugin.

Hotels

The group was split in two hotels. They are not exactly the same quality, but both were ok and quite close to each other.

• Hotel América – I didn’t get to see the rooms, but the people who stayed there said they were ok for a three star hotel.
• Hotel Princesa – I stayed in this one, and even if the rooms are quite small they are clean. The people at the reception desk (Nestor) are really friendly, although they don’t speak english very well. You can check their website for pricing information.

Eating and party

As it is usual in Spain, there are many places where you can eat on a budget and the quality is good. If you like fish and sea food in general this is a great place to go to. Just walk around the center and try one of the many places that you will find.

If you want to play it safe though, we found a very nice place called “El Cortijo” on the street Rosalia de Castro 59. Pili, the waitress, took really good care of us. Ask her for help to know how much food to order and what are the best choices to cover the wide variety of regional specialities. She gave us some discounts for drinks at Marmara, a music pub on the same street walking towards the center. The average age was a bit high, but it was a fun place to finish the workshop.

Related posts:

1. Trip to Poznan, Poland Last week I was in Poznan, Poland for a workshop...

Related posts brought to you by Yet Another Related Posts Plugin.

I am aware that the code is not very good nor clean, but given that it was my first shoot at doing something with Processing and I was in a trial and error mode I think it is quite normal. I also don’t feel like spending time improving this code as there are other visualisations that I want to work on and where I will have the opportunity of correcting the mistakes I am already aware of.

I just wanted to mention a couple of things:

• Processing does not support the declaration of enums inside a pde project. If you declare them in an external Java file you can use them in the pde. That is why the code looks weird passing String to define the type of dataset, and having all those cascading if-elseif statements.
• The code could benefit from a more OO approach and use polymorphism to avoid some repetitions. I promise I will do better next time

As usual, feedback and new ideas are very welcome

Related posts:

1. United Kingdom’s Met Office dataset preliminary analysis: The R script As promised, I want to share with you the R...
2. United Kingdom’s Met Office dataset preliminary analysis The United Kingdom’s Met Office recently released temperature data for...

Related posts brought to you by Yet Another Related Posts Plugin.

First of all, you need to download the dataset. I don’t want to hotlink directly the MySQL dump file that this guy produced, so just go over to his site and download it from there. In the meantime you can also see the perl script he wrote to convert the original dataset to the MySQL dump that we will be using.

Once you have downloaded the MySQL dump, uncompress it and import it into your MySQL database. In my case I created a database called “ClimateChange” and granted all rights on that database to a user called “ClimateChange” and without password.

Once you have your data in the database, you can start using R with this script available on GitHub.

Related posts:

1. United Kingdom’s Met Office dataset preliminary analysis: The Processing script After posting yesterday the R script that generates the averaged...
2. United Kingdom’s Met Office dataset preliminary analysis The United Kingdom’s Met Office recently released temperature data for...

Related posts brought to you by Yet Another Related Posts Plugin.

As lately I have been trying to learn how to use R and Processing, I decided that I would give a try to this dataset.

The first thing that I tried to do was to understand how the stations of the dataset are geographically distributed.

Even if the projection is not very friendly, it is possible to recognise the main mass bodies on Earth. The density of stations is non uniform with some areas over represented and some areas under represented. This might affect the stability and validity of global averages over time.

Next I was interested in seeing how many data was available each year, so I did a quick plot:

It can be seen how the number of measurements increased dramatically in the middle 20th century. But what caused that sharp increase in the amount of data?

Checking the evolution of the number of stations over time, we get our answer.

It can be seen how the number of stations has increased over time for each region.

• Artic represents the data recorded by stations north of the Artic Circle.
• North represents the data recorded by stations between the Arctic Circle and the Tropic of Cancer.
• Tropic represents the data recorded by stations between the Tropic of Cancer and the Tropic of Capricorn.
• South represents the data recorded by stations between the Tropic of Capricorn and the Antarctic Circle.
• Antartic represents the data recorded by stations south of the Antarctic Circle.

After understanding a bit better the evolution of the number of stations, I was interested in trying to see if I could find any meaningful pattern in the temperature data. So first I did an exploratory plot with the average monthly temperature for each region.

The first thing that caught my attention is the seasonal variation of the temperatures, and that displaying them in a scatter plot makes it unintuitive to understand that the right end of a plot is connected to the left end of the same plot (December – January). Then I decided to give it a try using polar coordinates.

Temperatures are represented radially, the angular magnitude corresponds to the months in a calendar year, while colors represent the years. The fact that the ellipse is not centered shows the seasonality of the data.

Then I decided to try to get one step further and try to show in an animation the temporal evolution of this data, and with my first Processing script ever, I created the following animation.

Global average temperature evolution 1701 – 2009 from Miguel Eduardo Gil Biraud on Vimeo.

Just to finish the preliminary analysis, I decided to check the evolution of the temperature averages over time, and to do that I did the analysis for the north region.

This graph shows a clear increase in the yearly temperature averages in the last 50 year! In a similar way that the visualization done by EagerEyes does. But is it the real story?

This is the same plot as before, but we have added in a color coded scale the mean latitude value for each measurement. The fact that the number of stations in the dataset changes over time, brings the mean latitude of the stations south (almost 10 degrees). Therefore, not all the temperatures have the same reference level. All in all, the chart is a case of apples to oranges comparison and it is telling a misleading story. If we plot explicitly the mean latitude variation, and the mean temperature variation, we can see that the variations follow each other.

This relationship between both magnitudes can be measured by the correlation factor, and in this case it is -0.7115842. Even if correlation does not imply causation, it should be a clear indicator for anyone to pay extra attention to the way in which the manipulate and present the information, as it is very easy to produce visualisations that will support a given idea even if the data says something different.

For the rest of the regions the correlation between temperature and latitude mean values are:

The difference in sign for the North and South are due to the fact that latitudes have opposite sign. While in the north lower latitudes bring the position closer to the tropic (higher temperatures), in the south this effect is achieved with higher latitudes.

In the next days I will post the R scripts I used to analyse the data, as well as the Processing program so that you can reproduce this analysis.

In these graphs, climate change cannot be seen, and it is meant as an exercise to illustrate how easy it is to produce plots that are misleading. Unfortunately, as of today, I lack the skills to reproduce the analysis that have been published in peer-reviewed papers with this dataset, but if you know how to do it, please go ahead and show us! I am eager to learn

As usual, I will be very grateful for any comments you have about how to improve the visualisations and the analysis.

Related posts:

1. United Kingdom’s Met Office dataset preliminary analysis: The R script As promised, I want to share with you the R...
2. United Kingdom’s Met Office dataset preliminary analysis: The Processing script After posting yesterday the R script that generates the averaged...

Related posts brought to you by Yet Another Related Posts Plugin.

In this post you will probably not find anything you couldn’t find anywhere else, as we did, but having it all in one place might prove useful and save you some time.

During the upgrade the mdam.conf file was overwritten with a default version of it, and the array was not automatically mounted anymore after reboot. The solution was found in a thread in the Ubuntu forums.
mdadm --examine --scan --config=mdadm.conf >> /etc/mdadm/mdadm.conf

The locale files got changed and we started to see lots of messages like:

perl: warning: Setting locale failed.perl: warning: Please check that your locale settings:
LANGUAGE = “es_ES.ISO-8859-15@euro”,
LC_ALL = (unset),
LANG = “es_ES.ISO-8859-15@euro”
are supported and installed on your system. perl: warning: Falling back to the standard locale (“C”).

The solution was found again in the Ubuntu forums. You have to edit /var/lib/locales/supported.d/es and make it look like:

es_EC.UTF-8 UTF-8
es_CL.UTF-8 UTF-8
es_DO.UTF-8 UTF-8
es_HN.UTF-8 UTF-8
es_PY.UTF-8 UTF-8
es_PR.UTF-8 UTF-8
es_NI.UTF-8 UTF-8
es_ES.UTF-8 UTF-8
es_PE.UTF-8 UTF-8
es_VE.UTF-8 UTF-8
es_GT.UTF-8 UTF-8
es_CR.UTF-8 UTF-8
es_BO.UTF-8 UTF-8
es_US.UTF-8 UTF-8
es_AR.UTF-8 UTF-8
es_PA.UTF-8 UTF-8
es_SV.UTF-8 UTF-8
es_UY.UTF-8 UTF-8
es_MX.UTF-8 UTF-8
es_CO.UTF-8 UTF-8
es_ES.ISO-8859-15@euro ISO-8859-15

Finally you have to rebuild the locales:

sudo dpgk-reconfigure locales

And last but not least, after the upgrades we were getting thrown into the busybox initramfs prompt. Apparently there is a problem with the evms and ACPI support, so we just disabled it, until we have further time to investigate the issue. The solution was found here.

Add into /boot/grub/menu.lst in the line that begins with #kopts

acpi=noacpi irqpoll

and then:

sudo update-grub
sudo aptitude remove evms

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

If you have never heard of a Heinsenbug or a Schroedinbug, you might want to take a look at the Unusual software bugs page in Wikipedia. If you are like me, you will feel relieved to know that you are not alone when you have to deal with those kind of bugs and their names inmediately make sense.

As for the quotes, the initial one shared by @vicchi was:

“Nine people can’t make a baby in a month.” – Fred Brooks

but there are many more available in the following websites:

• Top 13 funny software development quotes
• 101 great computer programming quotes – and there are more than 5!
• 101 more great computer quotes
• Great programming quotes

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

We sampled a few pubs and I recommend the following:

• Brovaria: A nice restaurant/bar with its own brewery. They have wheat beer, normal beer and honey beer, plus a selection of good food.
• Dragon: A cool place with lots of wooden stairs around a central patio. It looks like a castle
• Johnny Rocker: Quiet place near the square. Good place to set a meeting point while enjoying a glass of Ziwiec or Lech (Polish beers)

You can see some of the photos of the city and the workshop:

Related posts:

1. Trip to Vigo, Spain Once more I had the chance to visit Vigo, a...

Related posts brought to you by Yet Another Related Posts Plugin.

During GEOSTAT2009 earlier this year, I had the pleasure to meet Professor Roger Bivand, a british geographer teaching in a norwegian economics schools who was kind enough to spend a week in Croatia sharing his knowledge on geostatistics with us.

He is one of the authors of the sp package for R and he has a very good overview of the state-of-the-art on geostatistics.

The last day of the course, I proposed him to run a short interview on my blog and he accepted. His answers came very fast to my mailbox, but it took me way too long to sit down and write them in this post.

I would like to thank Roger for his kind answers, and I hope you will enjoy reading them as much as I did.

As one of the main developers of the sp package , what was your motivation to do it?

It’s mostly described in a talk I gave at DSC’03 in Vienna although the actual sketch was made on an excursion at the StatGIS meeting later the same year, at which Edzer Pebesma and Virgilio Gómez Rubio were also present, and where Albrecht Gebhardt was a local organiser.

The idea is not so much to provide a single standard representation, but to be open to interfacing with other representations, so that exchanging data may be made easier.

Has SpatialData (and their related structures) become the standard to represent spatial information in R? If any, which are the alternatives?

The large number of packages that depend on or suggest sp indicates that others find the representations useful. Other representations include those in the maps, PBSmapping, and spatstat packages among others – most are listed on the CRAN “Spatial” Task View.

Is it possible to contribute to the sp package or it is a piece of work that is complete and just maintained by a selected few?

The package source is (still) hosted on the r-spatial project on sourceforge. Contributions are welcome, but their incorporation is the responsibility of the authors, so it may be suggested that a separate package is more appropriate. I guess “select few” is not inappropriate, but that’s just the way things have happened.

What would you like to see implemented in the sp package that is still not there?

After four implementations of rings and polygons as something like a shapefile, it might be worth looking at an OGC Simple Features representation instead, otherwise nothing much on the TODO list.

Lately there has been a lot of talking about R (it got even an article on the New York Times) Have you noticed an increase on the interest on the sp package and the spatial capabilities of R?

There is a good deal of interest, witnessed by the number of subscribers on R-SIG-GEO, and offlist email traffic. But interest in itself isn’t interesting, it’s the quality of the new ideas and good bug reports that keeps the momentum up.

As the person who runs the R-SIG-GEO mailing list, how would you describe the community there? What are your plans (if any) for the mailing list?

There are an increasing number of “helpers” – subscribers who both start new threads and reply to questions in existing threads – it would be nice to see that trend continue and strengthen. It seems important to help subscribers to express themselves clearly – some problems are seen rather differently in other disciplines. I don’t have plans beyond trying to answer well-framed questions as they are posted, something which can take a good chunk of my day. I can feel guilty if someone has to wait, or doesn’t get an answer, but know that this isn’t a sustainable model, others can often answer better than I could.

As a geographer working in an economics school, you have a solid background on geography and statistics. What would be the baby steps that you would recommend to people lacking such a background to get initiated in the field of geostatistics up to the level in which they can follow the ASDAR book?

This is hard to answer, as people vary a lot. Some like to study a book or books – many are indicated in the ASDAR book, at different levels. Others are driven by a research question, others again learn through examples. Probably the only shared characteristic would be that these are things you can’t really get unmotivated people to grasp, but if someone is motivated, they’ll find their way in somehow. And enjoy it, after all, it is supposed to be fun, isn’t it?

Geostatistics is a relatively recent field and some of the calculations have been only possible in the last years thanks to the development of cheap computing power. Supposing that computing power and memory are not an issue… what do you think would be the next developments in the geostatistics world? (4D interpolation/prediction? local geostatistics?)

Hard to say, but probably more model-based geostatistics and conditional simulation (making the best use of limited data). Beyond that, I think that before 4D or even 3D, we should expect more movement on geostatistics for non-continuous data (count data and presence/absence data), something like GLMM models, and most likely hierarchical models capable of doing something sensible with zero inflation, which dogs many research problems. In fact, getting statisticians involved in helping design data collection is more likely to yield better results than more compute power.

One of the biggest challenges we need to resolve is related to information visualisation. The outputs of the calculations need to be easily understandable by others. Do you think that R is the right place to develop new visualisations or would it be better to export/communicate the data to other applications in which visual designers feel more comportable?

If we are thinking about analytical and exploratory visualisation, then yes, R is an appropriate place, not least because a lot of the research on statistical graphics and data visualization is done in R. Dynamic linked graphics with zooming are not easy, and will take time to do in a cross-platform way, so maybe there, other software environments may be prefered. The current graphics model in R is under constant development, with the newer achievement including anti-aliassing and proper use of the alpha channel where possible. This is also linked to attempts to provide proper graphics support for UTF-8, which is harder than one might think. A lot is going on at the moment in these areas, but time will tell which innovations succeed and diffuse.

Related posts:

1. GEOSTAT 2010 Last year I attended GEOSTAT 2009, and it was a...
2. GEOSTAT2009 Last month I had the chance to attend GEOSTAT2009, a...

Related posts brought to you by Yet Another Related Posts Plugin.

If you want to try it yourself, at your own risk, you can follow the instructions available in the following links:

• How To Jailbreak iPhone OS 3.0 On Windows and Mac Using redsn0w
• How to Unlock iPhone 3G Using ultrasn0w

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.