Segmented Regression of GISS Temperature Trends

In this post I show how to use the R strucchange package to prepare a segmented regression of the annual GISS temperature anomaly data series.

Don’t Try This With Excel

In this post I show 4 charts of the same data to demonstrate  what Excel chart users are missing by not having a more powerful charting tool. This post,  building on my previous discussion of using factors for conditional formatting,  shows the potential advantages of plotting summary values and bounding area polygons . These analytical displays are not readily available to even advanced Excel users.

Temperature Anomaly Data Displays

Is there a single “best” way to display temperature anomaly data? The answer is obvious – NO! The best display depends on what we are trying to show.  Statistical charts compare one variable with one or more other variables.

Since our display option affects how we interpret the data, it is important to be clear on what we are comparing. In this post I want to show 3 ways to display temperature anomaly data and the implications that the display method has on our interpretation of the data. I’ll use a map, a trend chart and a dot plot.

R Panel Chart Beats Excel Panel Chart

In this post, I show how to make a 3 variable time series panel chart in R. As an Excel panel chart pioneer, I can tell you that it is very difficult and messy to produce this type of panel chart in Excel.  The example R panel chart uses R’s a step chart format for one plot and R’s vertical line  format for the other 2 plots. Several of R’s dynamic capabilities, not inherently available in Excel, are also used. No helper series are needed and the chart can easily be regenerated each month as new data is available.

Here’s The R Panel Chart

First, let’s take a look at the chart to see what’s so good about R charting.

R Script to Automatically Chart Web Based Global Temperature Data

This post shows an R script to automatically generate a trend chart from web based global temperature data.  The R script allows me to update my plot each month as soon as the source file is updated. The plot is self documenting so that the chart reader can see the data period,  the overall trend line, slope for trend line; the last data point is highlighted and value given. Date stamp and name are included in the margin.

R Lattice Plot Beats Excel Stacked Area Trend Chart

Start of Series on Switch to R for Advanced Charts

I have been a long time Excel chart user, with dozens of techniques and tools for advanced Excel charts. As I pointed out in this post,  Excel’s multivariate data visualization limitations are severe. For me, the best strategy is to switch to R for advanced charts and continue to use Excel for my data analysis and simple charts. 

This post starts a series on my transition to R that may be of interest to those Excel chart users who struggle with Excel’s limited multivariate chart capabilities. I will post videos and provide source data files as well as R scripts on my ProcessTrends.com website for those who want to try R for themselves.

Working Example

In learning R graphics, I will be using the BP Statistical Review of World Energy – June 2008 report for data and examples of ineffective Excel like charts. BP does a great service for all of us by publishing their data rich report and workbook on world energy trends, an incredibly important topic from economic, environmental and security standpoints.  The 45 page report includes 8 charts types and a total of 26 charts. From a data visualization standpoint, 4 of these chart types are acceptable and 4 are ineffective.

BP seems to use Excel – PowerPoint or a similarly weak data visualization tool for their charts. The telling point is the reliance on doughnut, stacked area trend charts, stacked column charts and clustered column charts to show multivariate data. I have previously discussed BP’s chart selection in my Chart Doctor page.

Stacked Area Trend Charts

BP’s Statistical Review includes 11 stacked area trend charts. Let’s take a look at one of them.

This multivariate chart shows 3 variables: oil consumption (Y axis) by year (X axis) and region (stacked area). To add the 3rd dimension to this 2D display, BP stacked the regions with a stacked area chart.

We can see the North American and world total values clearly, however, it is quit difficult to interpret the specific trends for the 5 other regions because their baselines change over time.

I’ve made this R lattice chart to show the oil consumption trends for the 6 regions. Notice that I have added the 1965-1982 data and have not included the worldwide total consumption.

This lattice plot adds the 3rd variable by making a small multiple chart for each region. In this example, the small multiples have the same X and Y scales so comparisons are very straightforward.

The lattice plot shows the individual regional trends clearly, we can see that Asia Pacific consumption has increased dramatically, now equivalent to N American consumption. We can also see that Europe – Eurasia consumption peaked in the late 1970s and has been drifting downward since. We see similar local peaks in N America and Asia Pacific consumption in the late 1970s. We can also see the smaller market share and increasing trends for Africa, S & C America and Middle East.  

The R lattice plot lets me see both the trends and market share for each region much more clearly than the Excel like stacked area trend chart.

Excel’s Missing Multivariate Chart Capabilities

Why use stacked area trend charts? Did BP’s charters use stacked area trend charts because their chart tool doesn’t have small multiples, trellis – lattice chart capabilities?   

Excel charters often use stack & cluster techniques on their their multivariate data because Excel doesn’t have built-in small multiple, trellis – lattice tools. To work around Excel’s multivariate chart limitations, I started making panel charts in 2006.  Once I started looking into R, I quickly realized that Excel work-arounds, tricks, neat techniques aren’t enough. The tool does matter. You need real multivariate  graphical tools to effectively visualize multivariate data. Since Excel’s chart engine doesn’t support effective multivariate data displays, it’s time to switch to a tool that does.

After spending so much time mastering Excel, it just makes me wonder, “.. why the h*** doesn’t Excel have trellis – lattice capabilities? “ Tufte wrote about “small multiples” in his 1983 book, “The Visual Display of Quantitative Information“. That’s 25 years ago. Has the Microsofts Excel Team read Tufte, Few, Robbins, Cleveland? 

I’d like to hear what you think about multivariate charting with Excel.

 

 

 

 

Data Loss Aversion II – R Lattice Plot

Click to Enlarge

This post continues Jorges Camoes discussion on data loss aversion.  In my first post on this topic,  I used a dot plot to show the 1967 and 2005 values to summarize relative shifts in households by total household income bracket.   Derek, giving in to data loss aversion, used a logarithmic axis technique to show all data in intervening years between 1967 and 2005. Andreas Lipphardt used small multiples to show 1967 and 2005 values as well as the overall change by bracket. We can combine Dereks “giving into loss aversion” and Andreas’s “small multiples” approaches to show all data for the 9 series in a compact trellis chart by using the R Lattice package. I have a brief discussion on using R  for advanced charting here. This R lattice plot has several advantages:

• Shows all data  
• All plots share common X and Y axes, reducing axis labeling
• Plot uses banking to 45 to enhance visualization

The terms trellis, lattice and panel charts seem to be used interchangably, depending on which software was used to develop the chart. Small multiples is a more generic term that applies to Tufte’s approach of making a series of small, similar charts.
 

Excel has limited tools for effective multivariate charts, no trellis or lattice charts and no built-in small multiples capabilities.  While we can use clever Excel workarounds like panel charts or manually generated small mutiples,  I find that it is wise to move beyond Excel’s chart limits for multivariate charts.
 

R, a powerful and free statistical analysis and graphing package, has excellent multivariate charting tools. The R learning curve is well worth it for those Excel charters who want to move beyond Excel to the wider world of advanced charting.