Franchu’s Lair

During GEOSTAT2009 earlier this year, I had the pleasure to meet Professor Roger Bivand, a british geographer teaching in a norwegian economics schools who was kind enough to spend a week in Croatia sharing his knowledge on geostatistics with us.

He is one of the authors of the sp package for R and he has a very good overview of the state-of-the-art on geostatistics.

The last day of the course, I proposed him to run a short interview on my blog and he accepted. His answers came very fast to my mailbox, but it took me way too long to sit down and write them in this post.

I would like to thank Roger for his kind answers, and I hope you will enjoy reading them as much as I did.

As one of the main developers of the sp package , what was your motivation to do it?

It’s mostly described in a talk I gave at DSC’03 in Vienna although the actual sketch was made on an excursion at the StatGIS meeting later the same year, at which Edzer Pebesma and Virgilio Gómez Rubio were also present, and where Albrecht Gebhardt was a local organiser.

The idea is not so much to provide a single standard representation, but to be open to interfacing with other representations, so that exchanging data may be made easier.

Has SpatialData (and their related structures) become the standard to represent spatial information in R? If any, which are the alternatives?

The large number of packages that depend on or suggest sp indicates that others find the representations useful. Other representations include those in the maps, PBSmapping, and spatstat packages among others – most are listed on the CRAN “Spatial” Task View.

Is it possible to contribute to the sp package or it is a piece of work that is complete and just maintained by a selected few?

The package source is (still) hosted on the r-spatial project on sourceforge. Contributions are welcome, but their incorporation is the responsibility of the authors, so it may be suggested that a separate package is more appropriate. I guess “select few” is not inappropriate, but that’s just the way things have happened.

What would you like to see implemented in the sp package that is still not there?

After four implementations of rings and polygons as something like a shapefile, it might be worth looking at an OGC Simple Features representation instead, otherwise nothing much on the TODO list.

Lately there has been a lot of talking about R (it got even an article on the New York Times) Have you noticed an increase on the interest on the sp package and the spatial capabilities of R?

There is a good deal of interest, witnessed by the number of subscribers on R-SIG-GEO, and offlist email traffic. But interest in itself isn’t interesting, it’s the quality of the new ideas and good bug reports that keeps the momentum up.

As the person who runs the R-SIG-GEO mailing list, how would you describe the community there? What are your plans (if any) for the mailing list?

There are an increasing number of “helpers” – subscribers who both start new threads and reply to questions in existing threads – it would be nice to see that trend continue and strengthen. It seems important to help subscribers to express themselves clearly – some problems are seen rather differently in other disciplines. I don’t have plans beyond trying to answer well-framed questions as they are posted, something which can take a good chunk of my day. I can feel guilty if someone has to wait, or doesn’t get an answer, but know that this isn’t a sustainable model, others can often answer better than I could.

As a geographer working in an economics school, you have a solid background on geography and statistics. What would be the baby steps that you would recommend to people lacking such a background to get initiated in the field of geostatistics up to the level in which they can follow the ASDAR book?

This is hard to answer, as people vary a lot. Some like to study a book or books – many are indicated in the ASDAR book, at different levels. Others are driven by a research question, others again learn through examples. Probably the only shared characteristic would be that these are things you can’t really get unmotivated people to grasp, but if someone is motivated, they’ll find their way in somehow. And enjoy it, after all, it is supposed to be fun, isn’t it?

Geostatistics is a relatively recent field and some of the calculations have been only possible in the last years thanks to the development of cheap computing power. Supposing that computing power and memory are not an issue… what do you think would be the next developments in the geostatistics world? (4D interpolation/prediction? local geostatistics?)

Hard to say, but probably more model-based geostatistics and conditional simulation (making the best use of limited data). Beyond that, I think that before 4D or even 3D, we should expect more movement on geostatistics for non-continuous data (count data and presence/absence data), something like GLMM models, and most likely hierarchical models capable of doing something sensible with zero inflation, which dogs many research problems. In fact, getting statisticians involved in helping design data collection is more likely to yield better results than more compute power.

One of the biggest challenges we need to resolve is related to information visualisation. The outputs of the calculations need to be easily understandable by others. Do you think that R is the right place to develop new visualisations or would it be better to export/communicate the data to other applications in which visual designers feel more comportable?

If we are thinking about analytical and exploratory visualisation, then yes, R is an appropriate place, not least because a lot of the research on statistical graphics and data visualization is done in R. Dynamic linked graphics with zooming are not easy, and will take time to do in a cross-platform way, so maybe there, other software environments may be prefered. The current graphics model in R is under constant development, with the newer achievement including anti-aliassing and proper use of the alpha channel where possible. This is also linked to attempts to provide proper graphics support for UTF-8, which is harder than one might think. A lot is going on at the moment in these areas, but time will tell which innovations succeed and diffuse.

If you have been following the news, you have probably noticed that many of the problems that the economy is facing stem from risky – when not deliberately illegal – practices. In that sense, it will be difficult to forget the subprime morgages fiasco, Ponzi schemes pulled off by hedge funds managers, among many others that the society is already aware of, and who knows how many more are still waiting to make their appearance.

I believe that greed and bad business practices carried out for a long time by people who were entrusted to protect the economy are at the root of most of the problems the global economy faces today. Lies and speculation wove the fabric that sustained the stock market and the banking system, and those are very dangerous threads to make something that can hold in one piece and resist the test of time.

That is why I believe that professional codes of ethics are a useful tool, not that much from a legal point of view, but as a reminder to the individual of what is acceptable and what is not, and provide a set of guidelines to rule ones day to day professional activities. As an example of what these codes of ethics look like, we can take a look at the IEEE Code of Ethics and the ACM Code of Ethics.

Let’s go through them quickly:

• ACM
  • Contribute to society and human well-being.
  • Avoid harm to others
  • Be honest and trustworthy
  • Be fair and take action not to discriminate
  • Honor property rights including copyrights and patent
  • Give proper credit for intellectual property
  • Respect the privacy of others
  • Honor confidentiality
  • Strive to achieve the highest quality, effectiveness and dignity in both the process and products of professional work
  • Acquire and maintain professional competence
  • Know and respect existing laws pertaining to professional work
  • Accept and provide appropriate professional review
  • Give comprehensive and thorough evaluations of computer systems and their impacts, including analysis of possible risks
  • Honor contracts, agreements, and assigned responsibilities
  • Improve public understanding of computing and its consequences
  • Access computing and communication resources only when authorized to do so
  • Articulate social responsibilities of members of an organizational unit and encourage full acceptance of those responsibilities
  • Manage personnel and resources to design and build information systems that enhance the quality of working life
  • Acknowledge and support proper and authorized uses of an organization’s computing and communication resources
  • Ensure that users and those who will be affected by a system have their needs clearly articulated during the assessment and design of requirements; later the system must be validated to meet requirements
  • Articulate and support policies that protect the dignity of users and others affected by a computing system
  • Create opportunities for members of the organization to learn the principles and limitations of computer systems
• IEEE
  • Accept responsibility in making decisions consistent with the safety, health and welfare of the public, and to disclose promptly factors that might endanger the public or the environment
  • Avoid real or perceived conflicts of interest whenever possible, and to disclose them to affected parties when they do exist
  • Be honest and realistic in stating claims or estimates based on available data
  • Reject bribery in all its forms
  • Improve the understanding of technology, its appropriate application, and potential consequences
  • Maintain and improve our technical competence and to undertake technological tasks for others only if qualified by training or experience, or after full disclosure of pertinent limitations
  • Seek, accept, and offer honest criticism of technical work, to acknowledge and correct errors, and to credit properly the contributions of others
  • Treat fairly all persons regardless of such factors as race, religion, gender, disability, age, or national origin
  • Avoid injuring others, their property, reputation, or employment by false or malicious action
  • Assist colleagues and co-workers in their professional development and to support them in following this code of ethics

There you have it, a simple set of rules, sensible and perfectly reasonable, and spelled in plain english free of all the legalese that nowadays permeates every corporate sentence. And still, how many times have you been pressured (by others or by yourself) to ignore or trespass the aforementioned good practices? I agree that it is very tempting to circumvent or ignore those principles when a critical deadline approaches or a superior asks you to do something in a non-orthodox way; but if you break any of those principles you are breaking the trust you have been given to carry out your task in a socially acceptable way. You have to be able to explain people why you are not going to take shortcuts and contravene those basic principles, and make them see that it would have consequences.

I like to believe that technology companies learnt a lesson after the dot-com bubble burst, and therefore are better prepared in the current crisis, but the young graduates that are entering the work force right now have to be the ones ensuring that the codes of ethics widely accepted are followed strictly, in order to outroot the bad practices from the past.

Llevaba tiempo queriendo escribir un post sobre geotagging, pero nunca encontraba ni el tiempo para hacerlo ni el momento adecuado en el que tuviese reciente un viaje para mostrarlo. Afortunadamente, este fin de semana estuve en el Valle del Jerte viendo los cerezos en flor y he aprovechado que tengo el proceso fresco para contarlo aquí.

Pero antes de entrar en harina, ¿qué es el geotagging? Pues es algo tan sencillo como añadir información de posición a los metadatos de tus fotografías. Dicho así puede parecer muy complicado, pero no es mas que asignar unas coordenadas (longitud, latitud, altura) a una fotografía, para que quede constancia de dónde fue tomada.

Aunque hay cámaras que hacen todo el proceso de geotagging de forma automática (iPhone y alguno más…), lo más fácil para empezar y no tener que comprar accesorios para la cámara es hacerlo en dos pasos: por un lado tomar las fotos y grabar la trayectoria que has seguido, y luego mediante un ordenador asignar a cada fotografía la posición correspondiente. Para correlar las fotos y las posiciones se utiliza la hora GPS y la hora de la fotografía, con lo cual es importante que revisemos que la hora que tenemos en la cámara sea siempre la correcta. Por ejemplo, yo llevo en la cámara de fotos siempre la hora UTC porque así no me tengo que preocupar de cambiarla cuado cambio de huso horario o cuando llegan los cambios de horario de verano/invierno.

Una vez descrito en qué consiste, os voy a contar qué equipo utilizo yo para el geotagging. Por un lado una cámara de fotos (para esto te vale la que ya tienes) y un receptor de GPS que almacene la trayectoria que estás recorriendo y del que luego puedas extraer las posiciones a algún formato conocido gpx, kml, tramas crudas nmea, … Yo utilizo el Holux M-241 que tiene el tamaño de un carrete de foto de los de antes en color amarillo Kodak y que puedes llevar cómodamente colgado en la mochila o al cuello. De autonomía va bien con pilas alcalinas (10 – 12 horas) aunque según me han comentado con recargables aguanta bastante menos. Se encuentra fácilmente en ebay y a un precio bastante asequible para su funcionalidad.

Cuando llegues al ordenador extraes el fichero de recorrido (si necesitas cambiar el formato puedes usar gpsbabel) y ya estás listo para empezar a hacer el geotagging. Para extraer la trayectoria puedes utilizar el software que te proporcione el fabricante cuando lo compras, o bien intentar usar el software BT747 si el chipset del GPS es un MTK.

Llegados a este punto, tendrás un conjunto de fotos y un fichero de trayectoria que puedes pintar en Google Maps / Google Earth:

View Larger Map

Con estos ficheros ahora toca hacer lo que es el geotagging propiamente dicho, es decir, asignar una posición a cada fotografía. Para ello, recurriremos a alguna herramienta que automatice el proceso. En mi caso utilizo GpsPhotoLinker para Mac. El funcionamiento es muy sencillo, cargas la ruta, cargas las fotos y te permite procesar automáticamente un conjunto de fotos

La posición se guarda en los metadatos de las fotos, que puedes ver viendo las propiedades del archivo, o si lo subes a Flickr en las propiedades de la foto como por ejemplo en esta foto. Ten en cuenta que si quieres que Flickr tenga en cuenta la información de geotagging se lo tienes que decir explicitamente en la configuración de tu cuenta ya que por defecto está desactivado!

Al final el resultado, visto en Flickr sobre un mapa es bastante cómodo para hacerte una idea de dónde está tomada cada fotografía.

Ya no tienes excusa para no recordar dónde hiciste esa foto tan chula! Esta es mi forma de hacer geotagging, si tenéis otra forma de hacerla, usáis algún otro programa, me encantaría que me lo contáseis