small medium large xlarge

Dave_gnome_head_isolated_pragsmall
15 Jul 2013, 03:42
Dave Thomas (389 posts)
  • In the US, NOAA provides hourly XML feeds1 of conditions at 1,800 locations. For example, the feed for a small airport close to where I’m writing this is at http://w1.weather.gov/xml/current_obs/KDTO.xml

    .

    Write an application that fetches this data, parses it, and displays it in a nice format.

    (Hint: you might not have to download a library to handle XML parsing)

  1. http://w1.weather.gov/xml/current_obs

Generic-user-small
06 Jan 2015, 11:43
Pierre Sugar (57 posts)

I probably didn’t follow exactly the task. I have rather choosen to use the webservice provided by NOAA. But neverthelesss it was a good excercise.

The result can be found here: The challenge on Github

Here a sample output:

$ ./noaa --datasets -count 5
Datasets
name                      | id      | mindate    | maxdate    | datacoverage
--------------------------+---------+------------+------------+-------------
Annual Summaries          | ANNUAL  | 1831-02-01 | 2014-07-01 | 1           
Daily Summaries           | GHCND   | 1763-01-01 | 2014-12-31 | 1           
Monthly Summaries         | GHCNDMS | 1763-01-01 | 2014-12-01 | 1           
Weather Radar (Level II)  | NEXRAD2 | 1991-06-05 | 2015-01-04 | 0.95        
Weather Radar (Level III) | NEXRAD3 | 1994-05-20 | 2015-01-01 | 0.95        

Metadata
limit | count | offset
------+-------+-------
5     | 11    | 1     

$ ./noaa --locations -count 5

Locations
name        | id            | mindate    | maxdate    | datacoverage
------------+---------------+------------+------------+-------------
Ajman, AE   | CITY:AE000002 | 1944-03-01 | 2014-12-30 | 0.6859      
Dubai, AE   | CITY:AE000003 | 1944-03-01 | 2014-12-30 | 0.6859      
Sharjah, AE | CITY:AE000006 | 1944-03-01 | 2014-12-30 | 0.6859      
Algiers, AG | CITY:AG000001 | 1877-04-01 | 2014-12-30 | 1           
Annaba, AG  | CITY:AG000002 | 1909-11-01 | 1937-12-31 | 0.9527      

Metadata
limit | count | offset
------+-------+-------
5     | 38497 | 1     

$ ./noaa --data --dataset GHCND --location CITY:AE000002 --from 2014-10-01 --to 2014-10-01

Weather Data
datatype | value | station           | attributes | date               
---------+-------+-------------------+------------+--------------------
PRCP     | 0     | GHCND:AE000041196 | ,,S,       | 2014-10-01T00:00:00
TMAX     | 381   | GHCND:AE000041196 | ,,S,       | 2014-10-01T00:00:00
TMIN     | 285   | GHCND:AE000041196 | ,,S,       | 2014-10-01T00:00:00

Metadata
limit | count | offset
------+-------+-------
25    | 3     | 1     
Generic-user-small
16 Feb 2017, 14:25
Marco (1 post)

My solution was triggered by the hint “you might not have to download a library to handle XML parsing)”. So instead of an xml-parser I’ll decided to come up with a solution using RegExp.

The solution is very basic. So no parsing of a command line, fancy formatting or error handling.

Here you go:

lib/weather.ex

defmodule Weather do

  def go! do
    print_weather_stats("KDTO")
  end

  def print_weather_stats(city) do
    city 
    |> Weather.Fetcher.fetch_weather_stats
    |> Weather.Printer.print_status
  end  

end

lib/weather/fetcher.ex

defmodule Weather.Fetcher do

	def fetch_weather_stats(city) do
	  city
		|> weather_stats_url
		|> HTTPoison.get
		|> handle_response
	end

	def weather_stats_url(city) do
		"http://w1.weather.gov/xml/current_obs/#{city}.xml"
	end

	def handle_response({:ok, %{status_code: 200, body: body}}) do
		body
	end

	def handle_response({_, %{status_code: _, body: body}}) do
		{_, message} = List.keyfind(body,"message",0)
		IO.puts "Error fetching from Weather Database: #{message}"
		System.halt(2)
	end	

end

lib/weather/printer.ex

defmodule Weather.Printer do

	def print_status(response) do
		for row <- String.split(response,"\n"), do: print_row(parse_row(row))
		nil
	end 

	defp parse_row(row) do
		Regex.named_captures(~r/<(?<key>.*)>(?<value>.*)<\/.*>/, row)
	end

	defp print_row(%{"key" => key, "value" => value}) do
		IO.puts "#{key} = #{value}"
	end

	defp print_row(nil) do
		# Ignore
	end

end

And here is the result:

iex(1)> Weather.go!
credit = NOAA's National Weather Service
credit_URL = http://weather.gov/
url = http://weather.gov/images/xml_logo.gif
title = NOAA's National Weather Service
link = http://weather.gov
suggested_pickup = 15 minutes after the hour
suggested_pickup_period = 60
location = Denton Municipal Airport, TX
station_id = KDTO
latitude = 33.20505
longitude = -97.20061
observation_time = Last Updated on Feb 16 2017, 7:53 am CST
observation_time_rfc822 = Thu, 16 Feb 2017 07:53:00 -0600
weather = Fair
temperature_string = 32.0 F (0.0 C)
temp_f = 32.0
temp_c = 0.0
relative_humidity = 92
wind_string = South at 4.6 MPH (4 KT)
wind_dir = South
wind_degrees = 180
wind_mph = 4.6
wind_kt = 4
pressure_string = 1019.5 mb
pressure_mb = 1019.5
pressure_in = 30.11
dewpoint_string = 30.0 F (-1.1 C)
dewpoint_f = 30.0
dewpoint_c = -1.1
windchill_string = 27 F (-3 C)
windchill_f = 27
windchill_c = -3
visibility_mi = 10.00
icon_url_base = http://forecast.weather.gov/images/wtf/small/
two_day_history_url = http://www.weather.gov/data/obhistory/KDTO.html
icon_url_name = skc.png
ob_url = http://www.weather.gov/data/METAR/KDTO.1.txt
disclaimer_url = http://weather.gov/disclaimer.html
copyright_url = http://weather.gov/disclaimer.html
privacy_policy_url = http://weather.gov/notice.html
Generic-user-small
10 May 2017, 19:35
Raymond Loranger (1 post)

First, I wanted to enhance the issues project by formatting the data into a real table (not just separators). To achieve this, I used various box-drawing characters. Then I decided to add colors to the borders, headers and data. I came up with a number of table styles depending on the borders and colors chosen. I then added command-line switch --table-style to dynamically apply the related table style to the data. Then I factored out all the table formatting code into a separate package called io_ansi_table.

Finally I created 2 packages (the 2nd one is the exercise solution) using the above package as a dependency:

Per Dave’s hint, I do not use a library for XML parsing, just regular expressions.

First I find all the station IDs of a given state like so:

# <a href="display.php?stid=(KCDA)">Caledonia County Airport</a>
~r[<a href=".*?stid=(.*?)">.*?</a>] # capture station ID
|> Regex.scan(body, capture: :all_but_first) # i.e. only subpattern
|> List.flatten # each item is [station_id] 

Then for a given station ID, I find all its observation values like so:

# <(weather)>(Fog)</weather>
~r{<([^/][^>]+)>(.*?)</\1>} # capture XML tag and value
|> Regex.scan(body, capture: :all_but_first) # i.e. only subpatterns
|> Map.new(&List.to_tuple &1) # &1 is [tag, value]
Generic-user-small
19 May 2017, 11:02
Matthew Fehskens (6 posts)

I used the issues project as my base for creating this one. I leveraged the CLI code from that project as well as some of the setup.

I used the Erlang xmerl library to handle the XML parsing and my output is similar to what is seen on the NOAA site (example)

For the XML parsing I pull every XML element and its text:

def get_xml_info(node) do
  %{ children: children } = get_xml_info(node, %{ children: [], tag_name: :root, text: "" })
  children
end

def get_xml_info([:xmlElement, tag_name, _, _, _, _, _, _, children | _tail ], parent_info) do
  current_children = Map.get(parent_info, :children)
  node_info = Enum.reduce(children, %{ children: [], text: "" }, fn(child, acc) ->
    Tuple.to_list(child) |> get_xml_info(acc)
  end)

  Map.put(parent_info, :children, current_children ++ [Map.put(node_info, :tag_name, tag_name)])
end

def get_xml_info([:xmlText | attributes ], parent_info) do
  { text, _ } = List.pop_at(attributes, -2)

  current_text = Map.get(parent_info, :text)

  Map.put(parent_info, :text, current_text <> String.trim(to_string(text)))
end

And use the returned array of children to get data values. For ease of use, in the printing module, I transformed the children into a map, with the datapoint’s :tag_name as the key:

defp _create_element_map(elements) do
  Enum.reduce(elements, %{}, fn(element, element_map) ->
    Map.put(element_map, element.tag_name, element)
  end)
end

You can see all the code for it on my GitHub

You must be logged in to comment