Defining location has become important to our technology and daily life. But how do we get this information and how many resouces are out there to support the demand for greater coverage of location services? In this article we overview the current inventory of major global navigation satellite systems (GNSS) including US GPS, GLONASS, Galileo, and BeiDou/Compass.
We had previously published an article related to open GIS data sources around the internet that were free to use. We wanted to update that list to expose even more resources and provide any new information on sites we mentioned before. Here is the original Geospatial Data Sources Article to get an idea on everything we started with before writing this one. It's worth checking that one out as that post is pretty extensive. We are thinking about putting together a list of companies who sell high quality geospatial datasets as well but for now we hope you find use out of this list covering free and open data. There are many more and we will post more versions of this list down the road. Let us know if you know any that we do not mention that you would like us to list.
Public/Commercial Open Data
- ArcGIS Open Data - This ESRI resource boasts around 50,000 datasets from over 3000 organizations worldwide. Its definitely worth doing a search here to see if your dataset of interest is contained within this library. Export formats can vary but can be spreadsheet, shapefiles, or KML.
- GeoDeg Beta is a site that allows users to search for Points of Interest covering a large variety of categories. These POIs are from Wikimapia, Wikipedia, and OpenStreetMap. After searching for categories of POIs such as Skyscrapers or TV Towers the users can export those lists as KML, GPX, or TomTom ov2 formats.
- Geo Community GIS Data Depot offers free datasets as well as paid data. In most of the geographic areas they are consistent in offering the following types of data but can provide more in high profile countries such as the United States:
- Admin/Political Boundaries - 1M - E00 Format
- Digital Terrain Elevation Data (DTED) Level 0
- Geologic - 1M - E00 Format
- Hydrography - 1M - E00 Format
- Hypsography - 1M - E00 Format
- Landsat Enhanced Thematic Mapper Mosaics - 2000
- Landsat Thematic Mapper Mosaics - 1990
- Landuse/Land Cover - 1M - E00 Format
- Place name Gazetteer
- Transportation - 1M - E00 Format
- Utilities - 1M - E00 Format
- Standford GIS Library - Although we are featuring Standford's data in this particular link it should be noted that many colleges and universities have public resources that often contain free GIS datasets to pull from. It varies widely on what is provided and are often designed to give students a few baseline datasets to start their projects. For example Standford has Admin Boundaries, Population data, and transportation files.
- Humanitarian Data Exchange (HDX) is a hub for humanitarian data just as the name hints. Not everything in here is Geospatial friendly but formats could vary from csv, geojson, to shapefiles. Data can range from health reports, data on specific crisis events over the years, to refugee numbers in many countries.
- Protected Planet gives users access to a World Database on Protected Areas. It is a joint project of IUCN and UNEP. The web application shows where protected areas of the world are and even integrated content from the WDPA, photos from Panoramio and text descriptions from Wikipedia. Data is exportable from Protected Planet.
- Koordinates is a search engine for publically available datasets to include GIS formats. The application and Data is meant to serve users on a worldwide level but we will note that many of the datasets seem to be very focused around Australia and New Zealand. Either way its a good service to occasionally check on to see if they have something of interest.
- OpenCellID is the world's largest collaborative community project that collects GPS positions of cell towers, used free of charge, for a multitude of commercial and private purposes. The data is completely free and the coverage is worldwide. The dataset is in the millions so be ready to use a heavy GIS program like QGIS or ArcGIS if you want to do some analysis or use cloud storage in Amazon and Google to get it up where it is more usable for more applications. They do have an API available for that purpose as well.
Government Open Data
- The Humanitarian Information Unit (HIU) is part of the Bureau of Intelligence and Research, U.S. Department of State. Not all the data is Geospatially friendly but can still over a good resource for researching areas of Humanitarian focus. Many of their products are PDF exports. At the very least it maybe worth reaching out to the agency directly if you need datasets from a particular product they have published.The goals of the HIU is:
- Identifies key sources of geospatial and georeferenced data best suited to meet the information requirements of our consumers;
- Collects timely, verifiable, and relevant data utilizing an extensive network of information partnerships;
- Analyzes data using multi-agency expertise and applying proven technologies to determine significant trends and relationships; and
- Disseminates information of value to all levels of consumers, from national-level policymakers to operational field managers.
- GeoPlatform provides shared and trusted geospatial data, services, and applications for use by the public and by government agencies and partners to be:
- A one-stop shop that delivers trusted, consistent data and services
- Authoritative data to support informed decision making
- Reusable applications and services for governmental and nongovernmental use
- A shared infrastructure that can host your data and applications
- A focal point where governmental, academic, private, and public data can be visualized together to inform national and regional issues
- The US Energy Information Administration not all data here is geo friendly but the website offers a wide range of energy related topic datasets to include things like US Power Plants of various types around the country, pipelines, and refineries.
- Homeland Infrastructure and Foundation-Level Data (HIFLD) provides National foundation-level geospatial data (US Based) within the open public domain that can be useful to support community preparedness, resiliency, research, and more. The data is available for download as CSV, KML, Shapefile, and accessible via web services. Some categories of data that can be found here include Borders Boundaries, Communications, Emergency Services, Energy, Finance, Geonames, Mining, Transportation, and Water Supply.
This year's GEOINT Symposium was hosted in Orlando, Florida taking place from May 15-18. Compared to last Year's attendees which had approximately 5500 registered attendees and 1600 registered federal attendees, this year conference has around 3500 registered attendees and 265 exhibitors.
To date USGIF has accredited 14 collegiate geospatial intelligence certificate programs graduating 650 students with another 300+ in scholarships as they have gone through significant strives to enhance education in the Geospatial industry. In previous years they have tested out certification programs of different types in an effort to properly define and standardize qualifications for the unique set of skills required in the industry. As of this conference USGIF will officially launch its transparent and transportable Universal GEOINT Certification Program for the GEOINT Community. Certification program testing will begin in July.
The theme of this year's conference is "The GEOINT Revolution" focusing on a the massive transition of the Geospatial industry. In many perspectives this revolution can be seen with the massive adoption of cloud based processing and machine learning as they become more common in the community. Other changes are visible as the growing involvement and adoption of more small businesses over traditional defense contractors are taking place and the rising growth of small satellites that are expected to overwhelm the skies providing unprecedented coverage of the earth in the commercial spaces.
The Keynotes and main presentations are covered by USGIF and Trajectory Magazine. Those presentations can be viewed by clicking the link below.
While at the GEOINT Symposium we focused on companies who brought unique products that stood out from our experience as analysts in the industry. These companies could be big companies who have brought out an emerging technology or a small startup who has created a new disruptive product. Below is a list of companies we wanted to highlight along with descriptions of what they brought to GEOINT this year. As you read through our write-up on each imagine how each technology and application can be potentially tied together and what that means as we look to the future of geospatial analysis.
DataWatch has a product called DataWatch Monarch focusing on helping users format, structure, and organize data. Datawatch Monarch is transforms data by allowing users to drop a file or web page onto the canvas then it automatically begins to extract into a spreadsheet like format. The application allows users to aggregate data from various sources too. When working with DataWatch Monarch users create models that can be reused quickly to perform the same tasks for a multitude of disparate sources that are unorganized. Their application is unique in our perspective because it is one of the few that focuses on the struggles of an analytic workflow before actual analysis takes place. Many analyst's time is spent conditioning, formatting, and aggregating data to be more friendly for ingest into analysis applications. DataWatch's platform helps significantly with this process. For more about DataWatch go to http://www.datawatch.com/
DataFission is a search platform that is designed to make unstructured data searchable without using metadata. This search can be performed against large quantities of datasets qualified to be considered big data such as video, images, audio, and more. Below is our interview with DataFission talking about their service. Something of note is that they are working on integrating service access for their customers through the Amazon cloud as well. Find out more about DataFission at http://www.datafission.com/
Tom Sawyer Software
Tom Sawyer Software demonstrated their product to us of their advanced graphic layout and analytic packages. Their software processes data and is able to create a visualization and map of entities from the context. It displays a unique evolution in graph analytics compared to other applications performing similar functions. Their software could even be applied to dynamic data feeds to understand relationships on the fly. Watch the interview below and find out more about Tom Sawyer Software at https://www.tomsawyer.com/
IkeGPS produces products such as IKE and Spike which are designed to use smart laser measurement solutions to capture, record and export measurement data. The software and hardware combination is very affordable by industry standards and is available for use on mobile devices. Check out our interview with IkeGPS below and for more about IkeGPS go to their website http://ikegps.com/
Echosec provides a location based social media search and discovery giving users a clean and efficient user experience for understanding the social activity around an area of interest and to get alerted when new activity happens. Echosec is one of our picks to watch out of GEOINT 2016 as they develop their tool out further and to see how their alerts might evolve to influence other tools and applications.
PlanetRisk has a product called Advanced Intelligence Management System (AIMS)™ which focused on situational awareness and risk management for businesses. This level of expertise can dive deeper into topics such as travel, supply chain management, asset management, insider threat and social media monitoring to help their customers get alerted when their personnel or facilities could be threatened. Below is our video interview with PlanetRisk on their risk management platform.
PlanetRisk also announced that they now have a mobile app available for government customers hosted on the NGA App store: http://apps.nga.mil. This application provides government personnel with safety information while traveling abroad. This safety information includes Dept of State warnings and Embassy/Consulate locations. The information is streamed live to app users from PlanetRisk's Global Information Operations Center. For more about PlanetRisk check out their website at http://www.planetrisk.com
Notable Exhibit Floor Mentions
Here are a companies we thought had great products but did perform video interviews with
Diffeo was one of the vendors we talked to that stood out in the conference. They were hosting their booth in the small business pavilion which can be easily overlooked but well worth checking out considering the unique products that are coming out of small startups.
Diffeo is focused on extracting and connecting textual content to entities automatically. What is fascinating about the tools if that it works alongside the user while they are working in applications such as Outlook and can potentially be applied to various data streams by using their APIs such as news and reporting channels to give users insight while they are reviewing text. The Diffeo staff coined their application as a semi-autonomous research assistant giving recommendations working alongside users while taking advantage of machine learning algorithms to adapt the more the service is used.
A few announcements was that they were admitted to Microsoft's BizSpark startup community and that their service can work directly with Microsoft's OneNote to help dynamically train Diffeo as the users use OneNote. Additionally Diffeo is capable of performing research on the Deep and Dark web. For more information about Diffeo check out their website at https://diffeo.com/
Ursa Space Systems
Ursa is a US based company that provides day, night, and all-weather satellite imagery and extracts information from those images to create dynamic streams of analytic data. That information could be as simple as car counting to reflect the daily operations of an individual facility or ship monitoring by visually locating and identifying ships observed on imagery through image processing techniques. There are many other applications as well but the main highlight about Ursa is that they are performing feature extraction for the purpose of producing the next step in applied data analytics without direct influence by an analyst workforce behind the curtain. The company has long term plans to setup a synthetic aperture radar (SAR) constellation of satellites. Find out more about Ursa at http://ursaspace.com/
USGIF Golden Ticket Winners
USGIF selected up to 25 individuals to be fully sponsored and invited out to the GEOINT Symposium. The invite includes complimentary GEOINT Foreword and GEOINT Symposium registration as well as activities such as a mentoring luncheon, special exhibit hall briefings, and an invitation to the exclusive USGIF Chairman's Reception. We had the opportunity to interview three of the Golden Ticket winners to gain insight on their thoughts regarding GEOINT Symposium and the industry as a whole. Check out the interviews below.
The GEOINT Symposium remains just as vital to the industry as ever before and even more so to help government representatives understand the current capabilities presented by companies while gaining insight into the potential future of geospatial analysis from the showcase of emerging technologies as highlighted by the theme this year, Geospatial Revolution.
It is clear that applications are evolving to not only assist analysts in their workflows but are being developed to automate and understand analytic topics and methodologies to the point that traditional workflows will be re-imagined in the coming years. Technologies such as cloud computing and storage have laid the foundation for enhancements in machine learning. We will very likely see applications similar to Google Now, Apple's Siri, Microsoft's Cortana, Amazon's Echo, and IBM's Watson within the intelligence community designed to not just recommend to the analyst suggestions but intelligently anticipate what the analyst wants to see by understanding their behavior. Technologies like the ones seen from DataWatch Monarch, Diffeo, and Tom Sawyer's Software hint at the communities shift towards these developments. This will not only cut down on the workflow but increase the quality of analysis by revealing new relationships in data too complex for any single individual to find in a timely basis or relying traditional workflows.
As we see with companies such as DataFission and Ursa the traditional analyst task of identifying objects and facilities can be potentially be replaced by new processing techniques of Imagery itself. These applications have not yet revealed that they can perform the job of understanding why specific activity is happening or applying context to detected change in an area but they have revealed that technology is available now to at least identify facilities, activity, and change from automated methods while paving for applied models of behavior to act on that activity towards the next steps in the analytic chain. Potential of these paths will be revealed even more as the number in sensors in orbit are significantly increased in the next five years.
As support for more small companies to be involved and leveraged in the community it will be interesting to see how many startups and small companies come to the table beyond what we see today. The processes for getting small businesses to be more involved in government contracts has been complicated in the past but it has been improving. With that said the playing field is already changing as the entry into the market becomes more complex as the need for solutions transforms into an intricate demand of technologies that go above and beyond simple applications or analytic toolboxes.
We at Project Geospatial will continue to monitor events beyond the conference and track this revolution with a continued effort in helping you understand how these changes will affect you and how you can leverage these new technologies to make a difference in your business or analytic workflow. We didn't have time to cover everyone at the conference but we hope the applications and stories we did cover help paint a future of what's to come in the Geospatial industry.
The GEOINT 2015 Symposium hosted by USGIF returns and this time it is located in Washington DC. The symposium in 2013 was delayed due to budget impacts to the government until Spring of 2014 in which the conference was still called GEOINT 2013* but with an asterisk to reflect representation of the two years combined event. This year with over 5500 registered attendees and 1600 registered federal attendees, an increase of 10-15% from last year, the conference offers more opportunities for networking and geospatial awareness than previous years.
One Attendee, Trent Wakenight, highlighted many of the keynotes and their points through creative graphic recordings. Below are a few that summarize the keynotes...
To see the Keynote Videos for yourself USGIF has them posted here: https://vimeo.com/trajectoryonlocation/videos
NGA's Pathfinder Program was announced by NGA's Director Cardillo. Pathfinder will be an unclassified lab to answer key intelligence questions using open source data. The project will operate through a network of in-house labs, off-site locations and telework, all of which will be interconnected through a secure collaborative service. The GEOINT Pathfinder team will consist of a cross-discipline group of data scientists, application developers, open source researchers, methodologists and analysts culled from the agency as well National System for Geospatial-Intelligence and Allied System for Geospatial-Intelligence partners.
“We must drive the synergy of our new unclassified sources and our sophisticated classified sources to enable more exquisite insights and understanding on the new, higher open playing field,” said NGA Director Robert Cardillo.”This shift requires major changes in every aspect of our discipline. Classified sources, methods and networks will always have value in our agency and to our customers, but we cannot always view unclassified information as supplemental,” said Cardillo. “Moving forward the reverse is more likely to be true – that which is exquisite, but classified will supplement an ever broader and richer unclassified base.”
Activity Based Intelligence was not as much as a buzz word as last year but the concept is as strong as ever with different speakers and products aiming towards anticipatory collection of satellite imagery and interacting with Structured Observation Management (SOM) of object databases. Analytic Modeling programs are more common and slowly becoming a norm as contractors are aiming to create efficiencies in the workflow of analysts throughout the Intelligence Community.
NerVve - We stopped by the NerVve booth after how impressed we were after last year's demo to check to see if they had any updates. A quick reminder and background, NerVve is an In-Q-Tel funded company that made its name last year at GEOINT by demonstrating object recognition software for video and imagery libraries with machine learning on top that to improve the results over time. Overall they didn't have any major updates but we did inquire about the management of objects such as if they had a common object library. They said they do not and rely on each customer to have their users develop their own object models from scratch each time. This is one area NerVve could improve their business models by offering customers a common object model library to work off of.
Leidos had an interesting product called GLIMPS. It was based around using data sources such as the Global Database of Events, Language, and Tone database (GDELT) and using it to derive a score applied to a global grid indicating where high interest areas around the world are. The dataset was described as being updated annually and distributed as a service to help NGA determine where they might focus their analysis efforts in the following year.
ESRI teased their upcoming product ArcGIS Earth. At first glance this is ArcGIS Explorer overhauled but the business strategy is to scoop up the void about to be created by the deprecation of various geospatial products by Google to include Map Engine and Google Earth Enterprise. ArcGIS Earth will be a 3D light weight client that will be very KML friendly along with other formats such as CSV and Shapefiles. This is a huge change from ArcGIS Explorer which acted primarily like visualization program. Google Earth compared slightly better than ArcGIS with point and line creation, KML management, along with CSV import put it slightly ahead of ArcExplorer. This new product maybe a game changer for ESRI as Google Earth slowly leaves the stage with lack of support from Google and no major updates to its capabilities in years. According to their website, http://www.esri.com/landing-pages/arcgis-earth, the product will be released to Windows users first with plans for other devices and OS platforms down the road.
Orbit Logic showcased a new version of its mobile application, SpyMeSat. Before the application just showed where and when a satellite would be passing over your current location. With the updates the app now lets you order imagery of a 1km2 or a 2km2 area at the cheap price of $20-$40. It even allows you to task a satellite for $500. Users buy the imagery simply through Google Wallet on Android Devices. Traditionally satellite imagery cost anywhere between $1000-$3000 per image. Orbit Logic has managed to bring imagery to consumers while bringing more market value to archived imagery.
Mappt was kind enough to allow us to interview them on their product. They offer a mobile solution to mapping that allows users to create, edit, store and share spatial information while disconnected networks. Primarily designed for tablets a user can go out to remote locations and map their surroundings by dropping points or taking geotagged pictures of the area. Mapping data created by the application can be synced when the application reconnects to a network. Checkout more about Mappt's mobile application here: http://mappt.com.au/
Map Story, http://www.MapStory.org, provides a free and open source platform for organizing data about how the world changes over time and space. Users create webmaps of from uploaded Geospatial data. The platform allows them to visualize the data in a way that tells a story about the data and enables the user to embed the Webmaps in their websites or share them. Ideal for journalists, researchers, and Educators this service can be comparred to a non-profit open alternative to MapBox.
GEOMAKERS an collaborative community of volunteers and enthusiastic makers that dream, build and implement open source Maker’s projects that involve mapping, remote sensing, navigating, and understanding our world geographically. The group had a booth at GEOINT showing off some of their maker projects while encouraging passing participants to get involved. Maker communities are an excellent way to innovate on a private level but can lead to out of the box innovations in the industry as a whole.
Education and Training
This year USGIF beta testing their new certification program offering analysts who attended the conference two tests, one focused on Remote Sensing and the other for GIS. The tests are designed to help USGIF refine the certification process and gain efficiency in testing professionals on skill sets that mirror real on-demand professional knowledge base of industry experts. As they are continuing to refine their program they continue to ask for more beta testers. For more information on USGIF Education and Training or If you are interested in becoming a beta tester for their certification program you can contact Darryl.Murdock@usgif.org.
Next year the GEOINT Symposium will be back in Orlando, Florida at the Gaylord Resort. Our presence this year at GEOINT was low as we are restructuring our website and content but we look forward to seeing everyone in Florida and plan to cover more topics just as we did at previous GEOINT Symposiums.
NGA Pathfinder Press Release: https://www1.nga.mil/MediaRoom/PressReleases/Pages/2015-07.aspx
In the first year's of Android smartphones applications such as Where, Gowalla, and Foursquare were the vanguard in providing a checking service that not only provided a useful utility purpose but also made the experience fun. It didn't take long before Google made an extension out of Google Maps called Latitude which had huge advantages in live tracking contacts and pulling one's own historical data whether it be manually or through an API that once existed. Social networks such as Facebook eventually made their own check-in service, Facebook Check-in, creating a more social experience allowing a user to check themselves into a location but also checking who is with them at a location or an event similar to Foursquare. Sometimes applications like Yelp include a checkin service but not as their primary feature.
The Where app was one of the first geospatial directories to look for nearby events and places relevant to the users interests locally when Android first came out and Google Maps mobile was less encompassing in features. The user could quickly find local yellow book listings, restaurants, and other businesses based off category or search.Where had a checkin service which helped in giving a User local ads and deals around the user along with a variety of other options. The checkin feature in the "Where" application was hidden and became lost in the depth of features their application had. Overall it was a very good application but Where was acquired by Ebay/Paypal in 2011 and eventually faded into history no longer available to download.
Status: Acquired in 2011 - App No Longer Available
Gowalla was fun application to checkin because it was about awarding users badges for checking in to certain types of places often or a variety of places. In addition users or organizations could make their own checkin challenges to submit to the public to achieve custom badges, such as checking into every country at Epcot to achieve the "Around the World" Badge. This was also a business model for them as they could get companies to create challenges for checking in to certain restaurants or businesses. At the time is was looked at as a competing service with Foursquare as they both emerged as similar start-up checkin services the same year. Sadly the fun checkin service Gowalla was bought by Facebook in December of 2011. The application was dissolved and forgotten and the acquired team team went on to work on the checkin service at Facebook.
Status: Acquired in 2011 - App No Longer Available
Google Latitude had a variety of unique capabilities. It not only showed the users's location on the map but also the users contacts once they were invited and accepted to share location data. The application also stored the history of a user's location data allowing them to pull the data historically through a web interface and even export the data as a KML viewable in Google Earth. The service allowed for a live KML service for each user depending on the shared permissions allowed a huge flexibility in how this data can be used. Google created an API for latitude which allowed for other mobile applications such as Latify to standup which creating a unique way to use the live and historical data pushing it to Fusion Tables to do even more. The checkin service worked alongside the live tracking to actively make aware the users location relative to contacts using the same application. At one point Google even created a unique statistics page connected to the historical data page which a user can view all the checkins and view a graphical visualization of where they go the most, Airport connections they've made, and a fun chart in how far they've traveled relative to the distance from the Moon. Google Latitude was a baked well put together product which eventually was stripped and integrated into Google+ for their location sharing service. Checkins became directly integrated with posting on Google+ and the API for Google Latitude disappeared completely. Users can still pull historical data from the location history page which includes data from the current Google+ integration but this doesn't include checkin data. (https://maps.google.com/locationhistory/b/0)
Status: Checkin Features and Location Sharing Transferred to Google+
Facebook offers a straightforward checkin service that allows the user, similar to Google+, to checkin as a post in Facebook. The post can include a picture or a description of activity. A unique feature to checking in just like on Foursquare is that you can include who is with you, adding friends from your contact list, which checks them in as well. Other than being a social checkin service sharing the event with your contacts on Facebook checking in on Facebook as no added value besides adding to the history of your activity in your timeline. If this checkin post was made public then a developer could extract the checkins to a map using the Facebook API applying value to third parties not necessarily the user.
Status: Operational and remains consistent
Yelp's application helps users find places nearby. Yelp is known for their user driven review system and their mobile application is an extension of that. Checking-in using Yelp allows you to share your experience with your contacts on Facebook and Twitter. Another form of checking in is using Yelp's feature to reserve a table at a venue using OpenTable, a separate application but in this case its reservation services are integrated into Yelp.
Status: Operational and remains consistent
Foursquare Old Logo
Foursquare New Logo Finally we have Foursquare. With Gowalla and Where dropping out Foursquare has risen above the rest as one the premier checkin apps for mobile. When Foursquare started out, their features were similar to Gowalla in that they offered badges for completing certain types of checkin accomplishments like checking into a Bar ten times. Like Gowalla Foursquare the application experience was wrapped around the concept of checking in. However, what stood them out was their business plan involved advertising local businesses. Local businesses could promote deals based on if someone checks into their establishment and if the user is someone who has checked in the most above everyone else they are labeled "The Mayor". With that titled the user maybe entitled to additional deals premium discounts or deals at that establishment. Another feature was the competitive point system which added to the Mayorship on a more open scale but you can compete for points on whoever checks into places the most between you and your contacts. That Gamification using badges, points, discounts and deals based on checkins, and the Mayorship feature quickly made Foursquare the premium Location checkin based service. Users are also able to export their checkin data as RSS/KML/ICS. Developers have an API they can pull from as well allowing them to take advantage of the Foursquare checkin service in their own applications.
Foursquare has come along way since its launch in 2009. In the last week the company has separated their application into two separate application aimed at serving two different purposes; Foursquare and Swarm. Checkin features have now been watered down into Swarm which advertises on Google Play to be a more focused on the social aspect of checking in with your contacts as mentioned with the following features:
- Quickly open up the app to see who is out nearby.
- Checking in is faster and more fun than ever.
- Got an idea for something fun to do? Easily message all of your nearby friends.
You can still checkin with Swarm but its lost the appeal as it no longer has the popular features such as Mayorships and checkin discounts you previously had with the original Foursquare application. The Foursquare application maintains the name but has evolved into features dedicated to finding locations you are interested in based off of what they know about you. As the application learns more about what you like in different venues the searches begin to become more personalized. When you find an establishment you can view tips about the venue. If you leave tips or share your experience about the place you are at you earn "expertise". The service seems wrapped around the idea of finding local knowledgeable experts who use the application and rating them among their peers. In comparison its a different spin on Yelp reviews.
The new redesign and separation of Foursquare into two applications has come with much criticism. Many people don't like that they now have to use two applications instead of one. Others are wondering what happened to the Badges and Mayorship features which made Foursquare a popular service to begin with. Comments in both apps indicate they don't understand the purpose of using one or the other now. With these growing pains and user frustration many users are now being driven back to Facebook or Yelp for their checkin services.
Status: Checkin services are now transferred to Swarm App. Foursquare App itself doesn't have checkin feature any longer but now provides enhanced intelligent discovery and search based on user interests.
In summary, Checkin services still have a strong presence in the mobile world and there are many small scale ones out there I have not listed in this article but the above highlights the major ones that reach the masses. However, checkin applications are becoming less of a standalone application and more of a side feature alongside a social experience or discovery of local businesses to promote deals and events. This isn't bad but at the same time current versions of the remaining checkin services that exist no longer have the gamification or competitive feel that created that fun factor giving people that additional incentive to checkin even if their wasn't a deal offered or a friend in the area. This could be because the concept of checking-in is based off relying on a manual workflow where the users press a button to check themselves in. With refined location-based technology on mobile platforms checking-in could become more automated so it may not be as important in finding ways to encourage the users to checkin compared to taking advantage of their devices location data to give them what they are really looking for; local deals and social-geo interaction. As much as Checkin applications are moving in this direction they risk changing things to fast or too drastically for the user to understand the benefit while sacrificing what users liked about the old versions of checkin applications. Latitude into Google+ is a good example and so is the recent conversion of Foursquare.
Thank you for your time in reading this post. Please comment on your opinions on checkin services. I'd love to hear which service you use the most and why. What do you think of the new Foursquare/Swarm? What do you miss about checkin services in the past?
For the sake of this post we are going to focus on the mobile aspect of the term with an emphasis on Android devices comparing applications that handle geotagging a little differently between each other. First of all, let's answer the question what is a Geotagged photo?
"A geotagged photograph is a photograph which is associated with a geographical location by geotagging. Usually this is done by assigning at least a latitude and longitude to the image, and optionally altitude, compass bearing and other fields may also be included."
("Geotagged Photograph", 2014)
Before we get into applications that can take advantage of a Geotagged photos let's start by how to take one. At the very least you need your GPS turned on and some camera applications require you to go into the options and give the application permission to tag the location to your image as you take it so you will want to check this before you get started. I'm currently using a Nexus 5 and the location option for my photos is turned on at default. This means every time I take a picture a location is attached in the metadata of the image file itself; keep note as that fact is important later. However, with my Nexus 5 and other smartphones this fact varies as if I don't have the GPS turned on it will associate a best guess of where I might be which may default to the Cell Tower location itself, best guess based off cell tower triangulation, or if I have WiFi on the location of the router. If none of the above are available no location information will be tagged to the photo.
So if our goals was to just take a Geotagged photo then we are done right? Sure, but what's the value of tagging the location to images and what do we do with the photo from here?
Attaching location to information has numerous uses but for the common user adding this information is another useful way of helping you search for these images or organize them later. For instance many organize photos by date, event, or people around when the photo is taken. Location gives user the option to find photos based on a map, which in many cases, this is much more efficient in exploring the past rather than finding one particular photo through albums of thousands of photographs. By adding a temporal feature to the map a user can then explore photos taken through time even as events or overlaid at the same spot. This takes us into applications that can help you take advantage of the location data on your photos as each may offer you a different experience depending on what you would like to do.
With Instagram there are two ways to add a photo. One is to upload a photo directly from your phone and the other is to use the application itself to take a photo. Both options will eventually ask me if I want to add a location to my Instagram post and I believe this is more for privacy than anything else. Now this is what is unique about Instagram. Although it will read the location info on your photo to get an idea of where you took it Instagram is actually using that data to be attached to alongside your photo putting the geo data in the Post metadata not the online version of your image meaning once you upload your photo to Instagram the geo data is stripped.
Facebook, Foursquare, and Twitter are the same in regards to how they use Geo metadata. Initially they may take advantage of the information embedded in the image to add to the Post metadata but once uploaded to those applications the images are remade and the geo-data itself is stripped. For Facebook you can edit and change the location of an the image which has the benefit to adding location to images to which didn't originally have this information however the app doesn't actually embed that within the image itself just alongside it.
Why does this matter? One simple word, "Reusable". Well let's say you are using one of these applications as a primary storage for all of your images. You may want to download and reuse the images in something else or perhaps you may someday want to transfer your images to a new storage location. The fact that applications such as Facebook strip the original metadata hurts you as you are then missing information about those photos that make it special.
Should I still use Facebook and Foursquare for photos? For storing photos, No. Use of these applications is practically unavoidable though as everyone uses them for other purposes such as communication and sharing experiences with friends and family. There's nothing wrong with sharing photos to these apps but I would recommend storing your original geotagged photos elsewhere. Another reason is in addition to some of your metadata being stripped the images themselves are reconverted into a lower quality image for online efficiency for those applications. Users sometimes have the option to retain and store the full resolution image but that's a feature that may not be turned on default, just something to be aware of if you care about your photos beyond just sharing them.
Geotagged Data in image Google+ and Flikr are among the best mainstream photo storing applications. Both maintain the original metadata to include geographic location even after editing in those applications. Google+ is known more as a social tool but when you make a post and attach one your photos it retains the Geo-data and allows the user to attach a location to the post separately. This means when you view a post on Google+ from someone who posts from their mobile device you can potentially see where they made the post and when you click the image you can still see all the original image metadata. When you download your images from either Flikr and Google+ you retain everything. Google+ and Flikr just like Facebook also allow you to add location to photos that didn't originally have that info, but once again, the unique difference is these applications actually add the geodata to the image metadata itself which makes these two optimal online storage locations for your photos. These applications provide data transparency allowing you to view the metadata of the images, resusable, and flexibility to let you use them however you want. Flikr even has a feature where you can view all of the publicly viewable Geotagged photos in a map, https://www.flickr.com/map
Another great application worth an honorable mention is Panoramio. There used to be a mobile plugin downladoable from the Google Play store to allow you to upload geotagged images straight to Panoramio but these days its only accessible via desktop. Panoramio only takes Geotagged photos however it does let you edit the photos location after upload. In addition everyone's photos on Panoramio are publicly viewable and searchable on the map as seen to the left. I don't use Panoramio to store my images but instead to share my best geotagged photos.
Why should knowing how geotagging works important to developers?
Knowing how applications use geotagged information is vital to accuracy of the data itself and defining what the data says about the users or their content. Is the geographic location related to the post? Is it related to where the user posted about the event, where the user took the photo itself, both? If I'm looking for geospatial data on my users where do I find it? Am I looking at the images or should I look at something else. Google+ for example may contain multiple layers information that can be use in defining activity on a map. For example when creating an event that event may have a location attached, each post may have a location, and each image. Think of it has defining regional location all the way down to specific location telling a story as time goes on. As a developer this may help you define what activity you want to associate with users depending on where they are so identifying accuracy of geodata is important.
Please let us know in the comments what you think.
There are many more applications out there that take advantage of geotagged photos, this is just a few to help us define the topic. What is your favorite image storage application and why? Do you care about image location?
1. Geotagged Photograph. (2014). Retrieved from http://en.wikipedia.org/wiki/Geotagged_photograph
US Tornadoes 1950-2012
Finding open Geospatial data sources can be a challenge sometimes but there are plenty out there to take advantage of. Where do you find this data? What formats are they in? Are they commercially available or are they provided by the government? In this article we've come up with a list to get you started with a good foundation. Be mindful this is no where near everything but we hope what is below can lead in setting a good foundation.
US Government Open Data
- Among the most reliable and publicly available data comes from Government.
- Data.Gov - As we talk about Government data we should first mention Data.Gov. Although many departments of the government still act as standalone data source producers, Data.Gov is an initiative by the federal government to have one place where anyone can go to retrieve government open data. Since it first came online a few years ago, it has grown from a loose archive of static data files to a central nexus where multiple departments are finally connecting their open data in one spot. Departments such as the National Archives are now contributing this site. Developers can even access some data sources from the site using APIs. Data.Gov has a long way to go before it's links all the other Government departments into the initiative so you will still need to be aware of some of the other data source websites scattered.
- Geo Gov Data - Now completely merged with Data.Gov this was a filter for all geo friendly government datasets.
- US CensusDemographic Data - Every 10 years the US gathers data on a national level from every citizen and business in the United States for the purpose of tax accountability but the data itself has demographic value on a large scale. The Census Bureau even has APIs for developers to bring in the data into their own applications. Visualizing Census data is nothing new but merging the data with other sources has the potential to create powerful in-depth analytic maps and statistics that drill down to the zip code. This data has tremendous value especially looking at the data in historic trends and demographic expansion related to business geographically.
- Census TIGER Products - Topologically Integrated Geographic Encoding and Referencing (TIGER) Products is an extension of Census data in GIS ready formats such KML, SHP, and GDP
- Federal Communication Commission (FCC) - Doesn't expose too much data easily discoverable on their website but what it does have is files in reference to geographic market areas of radio/cellular licenses. The FCC's website has hints at having contributed to the data.gov initiative and more data might be found there
- USA.Gov -Points to a variety of government websites containing downloadable data essentially acting as a connector.
- USDA Forest Service/US Live and Historic Fires Detection - The USDA MODIS data is updates multiple times a day and provides information related to the latest fires around the nation. You can find archived fire data and live fire data in many formats with the most readily usable in KML network links.
- USGSEarthquake and Volcanic data - The USGS has one of the most open and flexible government datasets available to the public primarily related to seismic activity around the globe. Developers can find shape files, KMLs, and active data feeds to use within their applications.
- Shuttle Radar Topography Mission (SRTM) - The Shuttle Radar Topography Mission contains worldwide data set of topography giving a free and accurate model of the Earth available on demand.
- US Hydrography coming from USGS contains data set related to water levels around the United States. Stations around the country monitor bodies of water, streams, and rivers to indicate a variety of things such as floods and droughts. The data is updated periodically and in static formats such as GDB and shapefiles.
- National Geospatial-Intelligence Agency (NGA) Products and Services is responsible for offering a wide range of finished products for the public such as aeronautical charts, Maritime safety data and Topographical maps. NGA is also not a contributor on GitHub which can be found here: https://github.com/ngageoint
- Digital Mapping Charting and Geodesy Analysis Program (DMAP) Naval Research Laboratory (NRL) Mapping Branch offers a public tile server available through OGC standard web services such WMS, WMTS, KML, and WFS. The tile server contains nautical charts, Maritime Products (MODU, Broadcast Warnings, and ASAM), and aeronautical charts.
- NASA Visible Earth is a catalog of images of earth which are more end products but can be useful in some scenarios. The data sets are easily searchable and at the very least offer unique snapshots of the planet.
- NOAAOffice of Coast Survey GIS Data contains not only nautical charts but GIS data feeds such as WMS Services for maritime watches and warnings, limits and boundaries, and historical maritime map references.
- Environmental Protection Agency (EPA) Global Earth Observation System of Systems (GEOSS) contains numerous datasets, models, decision support tools, and programs that EPA manages, oversees, supports, or uses. Programs, tools and data cover the following categories: Air, Water, Land Preservation & Restoration, Healthy, Communities & Ecosystems, Compliance & Stewardship.
- US Fish and Wildlife Services Geospatial Services contain data related to the following: Coastal Barrier Resources (CBRS), Critical Habitat, USFWS, Ecosystem Regions, Landscape Conservation Cooperatives (LCCs), Migratory Bird Conservation, North American Bird Conservation Joint Ventures, National Wetlands Inventory (NWI), National Wildlife Refuge System (NWRS), Boundary data, NWRS Transportation Assets, Office Locations, and Regional Boundaries.
- National Park Service GIS Data doesn't host any specific datasets accept for perhaps downloadable maps of the National Parks around the nation but they do make available a wide list of links and resources that are related to the National Parks which much of it is managed by the USGS. Regardless, this linked assets provides a good starting point for anyone searching for NPS data.
- National Counter-terrorism Center (NCTC) - does not openly contain GIS data but reports about threats and activities are a good indication of events worldwide. NCTC does contain data available on request but not necessarily publicly available.
- Washington DC GIS Catalog contains one of the most publicly available and extensive GIS databases for a city and a model we hope many cities replicate. Everything from crime data to property data is available in a variety of GIS friendly formats.
- Pro-Logic GAPS Feeds (USSTRATCOM) - Not necessarily updated this site at the very least gives users an idea of what can be done with Google KML files related to government. Some of these files contain a good foundation of data sets as related to Homeland security and US based infrastructure.
- Social Explorer - "Social Explorer provides quick and easy access to current and historical census data and demographic information. The easy-to-use web interface lets users create maps and reports to illustrate, analyze, and understand demography and social change. In addition to its comprehensive data resources, Social Explorer offers features and tools to meet the needs of demography experts and novices alike." - from their webpage
- National Historical Geographic Information System (NHGIS) - "funded by the National Science Foundation (NSF) the National Historical Geographic Information System (NHGIS) provides, free of charge, aggregate census data and GIS-compatible boundary files for the United States between 1790 and 2012." - from their website
- University of Texas Maps - Using the University of Texas as an example many colleges provide open data for the public to take advantage of. Whether the data is available for each map varies. One example is the current Ebola Alerts Map.
- Free Map Tools - GIS analyst at home but don't have all the money spend on tools and applications then you may want to check out Free Map Tools. From their website they describe themselves as "An online resource that enables visitors to easily and quickly use maps in order to measure, search and overlay mark-up elements on maps for a wide range of useful applications."
- Old Maps - is a website to help users search for historical maps. The site then redirects the user to where they can purchase prints or download the data depending on the map. It should be noted that the David Rumsey Map collection is a major source of historical maps but you can also find those on other websites that provide those maps for free.
- Free GIS Datasets - This page contains a categorized list of links to over 300 sites providing freely available geographic datasets
Google Earth Based Data - Working with KML/KMZ
Google Earth as Displayed in Google Maps
Google Earth has been a major influence over the last decade in bringing Geospatial awareness to the common explorer. With that said KML and KMZ data are very useful and flexible formats for dynamic and static datasets not just for Google Earth but for online map applications as well. Here is a small list of some sources of those unique file types.
- Google Earth Gallery contains an archive of Google Earth ready maps and data provided by Google and contributed by many viewable through Google Map Engine along with a download to Google Earth.
- KML Factbook - This site is contains world wide demographic data and acts as an extension of the CIA worldfactbook to download the data in KML format.
- Google Earth Hacks - contains a library of unique Google Earth resources, files, and side applications to aide in what any user might need to do using Google Earth.
- Google Earth Blog - Links to Google Earth resources courtesy of an awesome blog worth subscribing to just as much as ours is. ;)
I can't end this article without mentioning the value of FusionTables which is a great resource for any GIS user. It's a free resource by Google in Google Drive that allows the user to upload GIS datasets in KML or Shapefiles (Some conversion maybe necessary, or even a spreadsheet of data into their online data table which allows the data to be quickly visualized in map format, aggregated with other FusionTables or accessed through data points such as KML or FusionTable APIs. This tool essentially allows the user to turn static date into dynamic data. Here are a few examples of Public Fusion Tables datasets.
There is an abundance of free GIS datasets out there and we are by no ways listing all that is available. An honorable mention should be social media APIs available to allow for the extraction of social chatter from applications such as Twitter, Google+, Facebook and more. We may get into that with a future article. Our goal is to at least increase awareness of what is out there for GIS specific purposes for starters. Do you know any government or commercial data out there please post in the comments. This list is very US heavy but we'd like to add more information on worldwide free GIS information available so please help us create awareness on what's out there.
GIS on mobile can mean a lot of different things. I like to think it depends on what the user's requirements are as there is no one source solution to GIS as a whole. I was inspired to write an article on the topic after an inquiry put out by one of our Google+ contacts asking what Android had to offer for apps relating to GIS recognizing that iOS has many. I think there are many applications that can fit the billet that are not even categorized in the GIS realm even though they should be. Just asking for something GIS related in general is a broad request therefore let's start with what our contact is actually asking for.
"Primarily something to take into the field for data collection. Something that will allow easy uploading of basemaps ahead of time, collecting point & line features using GPS. Definitely has to be able to work in an offline mode."
So if we interpret his request correctly he is looking for a mobile app that can do the following:
- Has the ability to work offline
- Can record GPS points or Tracks
- Can download or preload Maps
- Allows user to upload map files for basemap such as USGS Topo maps
- Can export data in a variety of GIS formats
The request has a very specific purpose as the individual would like to use a mobile application as an operator not necessarily as a desk GIS. The request doesn't involve any specific analysis then it could be more under the topic of a GPS application than GIS. None the less I began my search on Android where my first thought didn't bring me to a single mapping application that can satisfy all of the requirements but multiple apps that with one or two features here and there. I've used Google My Tracks with great success in recording GPS locations live and it gives you the option to sync/export the data to FusionTables, Spreadsheets, or some other type of storage device. However as stated My Tracks doesn't meet all the requirements. We still need an application that can not only record GPS, do it offline, but can allow the user to use or upload their own basemap data such as USGS Topo maps.
With that said my next thought was Google Maps/Google Earth Mobile. Google Maps does have the ability to let you download Map data but only Google Maps data. Google Earth, you may think what can that do? Well Google Earth mobile has had some significant updates over the course of the last year. If one uploaded their USGS Topo maps to Google Maps Engine you can login to that using Google Earth Mobile. What about offline? Well if you had KML file versions of those Topo maps and moved them onto your device ahead of time you can use them with Google Earth mobile too. I've gotten this to work with other KML files. Okay...sounds like Google Earth is the solution right? Nope, Google Earth does mark you GPS location but it doesn't record it so we would have to use a second app like Google MyTracks alongside Google Earth if we were to go this route.
My offhand knowledge has been exhausted at this point so what do we do? Go to the play store and do some hard searches. The application that seems to have the most potential meeting the requirements is Backcountry Navigator TOPO GPS. It costs $9.99 for the one time download but also offers a pricey subscription for Topo Maps. In addition it has a subscription option for making High Res Imagery requests from the DigitalGlobe archive. It very specifically states it is great for offline use while recording your GPS location as you go and import GPS waypoints from file formats GPX and KML. The reviews and ratings have this application at a 4.5 out of 5. With that said I concede that Backcountry Navigator TOPO GPS is the GIS application for this specific requirements requested. Will this satisfy our Google+ colleague? Time will tell as we have requested feedback as he tries the application out for his needs. Is this a GIS app or GPS app? I think it can be considered both depending on what the user intends to use the data for but specifically this is more on the side of GPS Mapping app.
Well that brings us back to the question what is a GIS application?
According to ESRI's website A geographic information system (GIS) lets us visualize, question, analyze, interpret, and understand data to reveal relationships, patterns, and trends. It just so happens ESRI has an ArcGIS app on the Google Play store that does just that. Unfortunately the reviews indicate that the application doesn't do well in without a good connection, let a alone no connection at all.
Other companies such as BAE Systems are using Mobile devices as a ground reporting tool in which data can eventually feed into more advanced GIS applications or databases. Perhaps this is all that is required to qualify as a GIS app. They currently have two apps on the Google Play store right now, GXP Xplorer Mobile and GXP Xplorer Snapshot which compliment each other. Xplorer Snapshot allows users to submit to the Xplorer Server with photos from their mobile device tagging their location. Xplorer Mobile allows the users to search from a vast archive of submitted data to include making queries to news feeds based off a desired location. The user can use the data from search results to create their own reports. BAE Systems showcased another Xplorer Addon application similar to snapshot designed for Google Glass at this year's GEOINT Symposium but is not ready for public release.
As a simple reporting tool many people underestimate their mobile devices in general. By combining the features of the device itself mixed with multiple applications you may have all the tools you need for a GIS Application for visualizing, researching data, recognizing patterns, and even creating a report. I already mentioned the power of using Google My Tracks alongside Google Earth. A few years ago I performed an experiment with a similar application to My Tracks called Latify in which I broadcasted my recorded location in KML as I took a road trip around the nation. The data was dyanamically displayed live on a website where in addition every time I submitted a Geotagged photo it would also broadcast those photos through KML to my website too.
There are many ways to mix and match mobile apps to meet your needs. Traditionally Blogger is known as a Blog management app but it can be just as powerful as a Geospatial Reporting tool. For example with Blogger, you can attach your location to every post, upload images to the post, write a report, and finally submit it to an online audience. What's even more unique is if you attach your location to a blog post you can extract that from the GeoRSS feed and even distribute and convert into KML to share out which means your blog can be a dynamic geospatial reporting tool. To put icing on the cake Google+ is now completely integrated into blogger to allow for a wider collaboration effort from social media or even a private audience if you decide to integrate the blog into a private Google+ page. However, if you have connection with your mobile device using this logic any social media website can become your own personal geospatial reporting tool as long as 1. You Geotag your posts and 2. You Geotag your photos. In the case of social media you may also have to make your posts public just to extract the data from the APIs.
I hope you enjoyed my thoughts about mobile Geospatial applications. There are ton of apps out there and when we get around to it we will list as many as we can but for now we want to know what you think. Do you have a GIS Application you enjoy using on Android or any mobile device? What features does it have that you like? What can it do better? Perhaps you have your own mix of apps that work well for specific Geospatial purpose. Let us know in the comments.