Erik on Software

New Book Review: "Do Something for Nothing"

New book review for Do Something for Nothing: Seeing Beneath the Surface of Homelessness, through the Simple Act of a Haircut, by Joshua Coombes, Akashic Books, 2021:

Copy provided by Amazon.

The subtitle to this book provides a great summary for what this book is all about, albeit from a limited perspective: "seeing beneath the surface of homelessness, through the simple act of a haircut." The introduction furthers this thought by stating that "this book isn't about homelessness, it's about humans and showing up for one another where we can." However, as a longtime consultant I also view this book as a phenomenal demonstration of the pro bono experience, which is distinct from traditional volunteerism: using one's professional skills to provide services to those who are unable to afford them.

I've personally performed both volunteer and pro bono services, but while I've served individuals in need as a volunteer, I've never served individuals in need as a pro bono consultant, instead serving mediators situated in-between myself and the end recipients. And my lack of experience in this area is why I especially appreciate what Coombes has to share in this book. Not only does the author actually carry out a professional service, cutting hair, but he spends time with individuals before, during, and after providing service to get to know each individual. And in my view, this is a great example of why transformational relationships are so much more powerful than transactional relationships.

In the world of consulting, I've often been advised that transactional relationships are inferior, but in the end it's always an expectations-filled, give-and-take business relationship. Not so with what the author demonstrates in this book, which puts to shame much of what I've seen during my lifetime, regardless of intent. Coombes spends some serious quality time with each individual, always trying to empathize with their circumstances and never outwardly judging. Some backstories provided by the author's "customers" are so especially moving that it has caused me to recount some of my own encounters with the homeless, and why I can't stand the imposters I've seen on city streets.

This book walks the reader through the author's meetings with 35 individuals on city streets across 14 global locations in the US, UK, France, Netherlands, Mexico, Australia, and India. It seems the fact that 40% of these individuals are located in the US speaks more to the interests of the author to visit from his base in the UK, partially because of his connections in the US, but this aspect of the book drives home the fact that homelessness exists everywhere, even amongst the affluent.

And as such, Coombes also surfaces the reality that homelessness can happen to anyone, even the affluent, by passing along the stories each individual has shared with him. David, for example, described his experience as "surreal". "There's no way I would have seen myself sitting here...no way at all. I've realized how we trick ourselves into believing that we're in total control of our lives, and we're really not. There's so much that can happen to a person that you ain't got no say, or no control over. It's frightening...it's really frightening. I see people walk past me and I get the odd person with a look of disgust on their face, but mostly I see fear on their faces, because they don't know how you got here, and they're scared they that could be here also."

In some cases, customers share glimpses of what their lives were like before homelessness. Emma, for example, doesn't ask for much. "Although sometimes I feel like saying, 'Forget this life, just end it,' I don't have the guts to do that, and I feel like that's basically the same thing as saying, 'Don't do it.' Because, all I need is a place or a room and I'll be happy. I could go in, close my door, and be back to normal life, doing normal things: waking up, having a coffee, taking the dog for a walk, coming home, doing some housework. I was doing embroidery before and getting really into gardening."

And in the midst of homelessness, there is also much optimism even whilst recounting past abuse. Two homeless individuals, Lavane and Nick, even found love for each other. "I'd been abused as a child at the hands of my dad...The last place I thought I'd find real love was on the streets, but then Nick came along...Before Nick, I was married for ten years and I had a really bad relationship. There was a lot of domestic violence. I left after he put me in a hospital for the second time. That's when I became homeless...I never met anyone who had loved me that way. He didn't care what people thought of him. He was unashamedly himself. I loved him for that - even when he smelled bad! He lived his life and had his flaws, like we all do, but his heart was good. He showed me a different way. He showed me how to live on the streets and how not to be frightened. It really gave me back my confidence. I really fell in love."

Before April 2020 when I first started cutting my hair due to salon closures, I had convinced myself that I enjoyed talking with those who cut my hair. While these weren't transactional relationships, money still exchanged hands and I don't recall ever hearing anything profound. In writing about his time with Tony, Coombes demonstrates a far different experience, and combined with the optimism so prevalent throughout, it provides a moving story. "Tony meant every word, I could feel it. They cut through more than any 'motivational speech' I've seen delivered onstage." As explained by Tony, "I've come a long way, man. My dad's an alcoholic. It wasn't pretty. My mom was everything to me. She died back in 2000. I know she's looking down on me and I take a lot of strength from that. I want to put out a message to inspire people that I'm a survivor. I'm living with cancer, on the street, and I'm still smiling. So be optimistic."

The story Dennis has to tell is perhaps especially moving. "It sounds crazy, but my thing is, even though I'm struggling, I think God put me here to help other people. You know, because everybody I've helped has got themselves together and moved on, and I'm good with that...I had seen everything - apartments, cars, and all that. So, I understand it's not easy to be out here every night. People with no money, getting high just to stop what's going on in your brain. I'm trying to learn to give without wanting - without wanting anything in return...Why is it so hard to want to see the next man do all right? If I see the next man down, I've gotta try and bring him up. Even if it's just a little bit. So he knows there's at least one person out there who's not just trying to get one over on him. That's where my heart is at. Maybe it's my age that's humbled me and brought me to that point. I dunno."

This reminds me of Romans 5:3-5: "...we also glory in our sufferings, because we know that suffering produces perseverance; perseverance, character; and character, hope. And hope does not put us to shame...". In my 2015 presentation on how to use one's skills to benefit a non-profit for a consultancy's annual technology conference, I walked through 16 takeaways I gained from my pro bono consulting work for 5 different non-profits. My fourth takeaway: "Encountering beautiful clients with relatively low prestige jobs is common." I highly recommend this book if you're interested in starting your own journey to meet such clients.

Centric Consulting CampIO 2015: Use Your Skills to Benefit a Non-Profit

My first attempt at assembling my thoughts around the topic of pro bono consulting, based on my past experiences working with 5 different non-profit organizations. These slides were presented at CampIO 2015, an annual conference organized for internal employees at Centric Consulting.

---

From the promotional materials:

Use Your Skills to Benefit a Non-Profit

What is pro bono consulting, and how is it different than volunteerism? During this talk, Erik will present these topics alongside what he has learned during his own pro bono experiences, with the goal of getting others interested in similar pursuits to ignite passion for the greater good.

---

       

Pro Bono: IEEE Chicago Section Statistician - Part 5

My last post in this series on my pro bono efforts outside of the workplace as Statistician for the IEEE Chicago Section mentioned that I walked the Membership Development Chair through the visualizations of my latest monthly presentation to the team, partially in preparation for his request of me to present to the executive committee. Unfortunately, the executive committee meeting that included a number of presentations by other individuals was already behind its already tight schedule at the outset, and so my presentation time ended up being reduced to 15 minutes.

While my presentation was originally intended to cover some of the visualizations and findings of the past year, delivery probably would have benefited from a scaled down scope that concentrated on data of recent months, including the month of March 2014 which was quite coincidentally the best month across the 8 years covered by the database in terms of total active membership net difference (membership additions less membership deletions), as well as April 2014 which was the first month during this same time period where total active membership "closed the gap" in YOY total active membership.

In the above plot, the line representing calendar year 2014 is on the bottom. Readers of this blog might recall that total active membership has been declining each year, but this trend has just been broken since total active membership is now above the same points of the previous year two months in a row. (Note that my plan is to overhaul these visualizations to account for color blindness. To be brief, two individuals - one of the executives discussed here, and one of my work colleagues in the workplace - recently mentioned that they are color blind. The first individual could not tell that the year 2014 was encoded with the color yellow, and the second could not distinguish between the blue and green colors I had chosen for Facts and Dimensions in the data warehouse model that I had built for the team.)

One positive result of my presentation to the executives is that the new Section Chair/President had a number of follow-up questions to ask later that same week which I was able to address. Essentially, what these questions encompassed was the ability to forecast/predict what the upcoming year is going to look like in terms of membership numbers. Questions such as this came as no surprise to someone in the trenches with the data. However, in response I emphasized that what I have been putting together for the team is descriptive analytics, not predictive analytics. No visualizations that have been provided to date should be seen as predictive analytics.

As explained to the new Section Chair/President, I could easily create what some might view as predictive graphs, such as trend lines etc, but I would be doing the Section a disservice by doing so, since these would not be accurate. In thinking about what helps predict future months, it really has a lot to do with individual members. It also has a lot to do with the economy and general industry trends, but these are fairly abstract concepts - even the government cannot predict unemployment rates and is always adjusting past months. In thinking about individual members - their demographics, education, fields of work, technical interests, etc - these are the core factors upon which models can be built to predict the types of individuals we are attracting and the types of individuals who choose to leave. Unfortunately, we do not have access to member credit ratings, but if we did, these might additionally help predict who will fail to keep up with their membership dues and go into arrears each year.

Assembling data sets created from the database in order to start capturing this information might be a start, but it would be a slow process because I would only be able to assemble data sets of the current month for which data is available. Once the data is updated for the next month, much of the information of the prior month is no longer made available in the database. The aggregate totals are still there, but not the data from which these aggregates were created. As discussed with the team, at minimum I would need the ability to query the database to determine whether an individual was active for a particular month in the past, but the historical member grades that the database tracks for each individual alongside the historical statuses of members (when corporate IEEE decides to provide this information) would only be a start. As discussed with corporate IEEE last year, essentially all of the data needs to be effective dated.

To provide readers a better understanding of the current state of the IEEE membership database, below are some screenshots of the Oracle Business Intelligence Enterprise Edition (OBIEE) 11g user interface. The first screenshot shows a small subset of the data available about me that is specific to historical member grade effective dates. Starting as a Student Member during graduate school, I was later elevated to Associate Member, Member, and most recently, Senior Member. The memberbership status field apparently indicates that I was in Active status during all of these time periods, with no other statuses.

Now take a look at the next screen shot, which contains similar items, albeit for another member that I anonymized by not including any personally identifiable data. In quering data for this individual, I also ordered by historical member grade effective date. As you can see, this member has a long history with IEEE, and was elevated to Life Senior membership status over 25 years ago. While the membership status change date for my entry above is the same as the start date for my first historical member grade, however, notice that membership status change date for this other individual is just a few months ago. So why does the membership status field indicate Inactive for all time periods? Because the database does not track historical member status and only records current status.

This particular data is very unuseful, unless perhaps the user only cares about current state, or is willing to keep track of changes over time in a separately maintained data set outside of the IEEE membership database. But notice another apparent anomaly. The membership status change date is one day removed from the most recent historical member grade termination date. While this data might be accurate, it seems very odd that the two dates do not line up with one another. Essentially, what this example seems to imply is that a presumably active Life Senior member went into Inactive status with one day to spare. And this anomaly does not even address the fact that this time period is the second of two as Life Senior member. Why are there two time periods? The database simply does not divulge this information.

In thinking about the models that can be built with predictive analytics in mind, consider the data associated with personal updates that LinkedIn provides to its members. For readers not famliar with this feature of LinkedIn, personal updates are essentially a text box via which members can share messages with their connections. For this discussion, the content of these messages is not technically important, but keep in mind that these personal updates are often used to communicate career related events and share blog posts such as the one you are reading right now. For illustration, I captured some screenshots of another LinkedIn feature labeled "Who's Viewed Your Updates" that is directly linked to the personal updates feature.

The above four screenshots, clockwise starting with the upper left screenshot, are intended to show who viewed three personal updates that had been shared in early March 2014. The first screen shot is a summary of the latter three screenshots, which are broken down by each individual update. Fortunately, the numbers add up. There were 65 total views by my connections of these updates (31 + 15 + 19 = 65), 4 total likes (1 + 1 + 2 = 4), and 1 comment. While this is basic math, notice that the plots that LinkedIn provides do not match these numbers. At some point, LinkedIn must have determined that there is a saturation point with the number of data points being depicted. Although this is a general observation, because this only seems to apply to the total number of views, note the number of likes and comments, which are accurate.

So what does all of this have to do with predictive analytics? The key word in this LinkedIn feature is "who" - "who" has viewed particular personal updates. Where is the "who"? If I hover my cursor above one of these data points, does a depiction of the associated LinkedIn connection materialize nearby? No. All that can be reviewed by a member here are counts. A member cannot actually take a look at who viewed their updates. With the assumption that I have the same number of LinkedIn connections in the future when another personal update is posted, is there any way for me to determine how many will view, like, or comment? Probably not. As colleagues have heard me say on a number of occasions over the years, anything is possible - but not everything is probable. Now consider a recent personal update that is separated in time from the previous three updates by a few months.

What, no data points? We already established that there is no "who" per se in the equation. And we know from the first example from a few months ago that the total number of 47 data points would not be included if these were depicted. But why are there no data points? Well, it turns out that LinkedIn has decided that unless there are likes or comments, all of the data points would simply be views and so these would not provide any value. Perhaps. But one can argue that only displaying a subset of total data points alongside total likes and comments can be misleading, giving the impression that the ratio of likes and comments to total views is actually higher than it actually is in reality.

What all of these observations have in common is that in order to predict, one needs to be able to drill down to the individual data points to understand the characteristics of each so that the likelihood of appearance of other data points in the future exhibiting similar characteristics can be ascertained. In looking at what is currently made available to LinkedIn members, one cannot determine the audience of personal updates. And while granular information about IEEE members is technically being made available for the current month during which the database is being queried, this information is not being made available on a historical basis. Going back to my plot of total member count from May 2014 above, we do know that increasing senior member count has had a big impact in recent months, and the database does keep track of the types of member additions and deletions that take place over time, but we need to understand the characteristics of members behind these numbers. Looking at total counts does not really show the big picture.

---

Pro Bono: IEEE Chicago Section Statistician - Part 1, Part 2, Part 3, Part 4

Pro Bono: IEEE Chicago Section Statistician - Part 4

This post is the fourth in a series on my pro bono efforts outside of the workplace as Statistician for the IEEE Chicago Section, a new position that I filled in September 2012 after the membership development team realized that ongoing data analyses and reporting was needed around declining enrollment.

As discussed in the last post, the bulk of my initial time spent on this work was fixing existing graphs that I had inherited, moving from Microsoft Excel to the R language, maintaining existing data sets, creating new data sets, and creating formal presentations that I started distributing on a monthly basis.

After creating re-runnable R scripts in early 2013 for the monthly graphs that the team came to expect, my attention turned toward creating new visualizations that seek to better understand the data, and I began to spend more time to communicate my analyses for the nonanalytical.

My first post discussed some of my initial thoughts surrounding my early experiences on the team, and presented one of the new visualizations that I created, a heat map that breaks down top technical interest areas of members which the Membership Development Chair began using in his marketing efforts.

In my second post, I walked through some of the issues in the existing graphs that I had inherited, and showed how I fixed them in Microsoft Excel and ported them to the R language with a consistent look and feel, as well as how I began including them in IEEE Chicago Section presentations.

My third post discussed the goal of the Membership Development Chair to "close the membership gap" that is currently increasing each year, and how these declines in membership are heavily the result of members going into arrears each February when annual membership payments become due.

Rather than closing the membership gap during the course of the year (i.e. during months other than February), I argued that the goal should really be to minimize the number of members who go into arrears due to missed payments, because on an annual basis a pareto chart would show that membership deletions due to arrears status exhibit by far the largest frequency of occurrence.

During the annual membership development summit meeting held in November 2013, I had the floor for a bulk of the discussion, and I emphasized that we should really be comparing to previous years, the reason for my inclusion of an increased number of visualizations to the mix.

Our membership numbers get hit hard one month of the year, and then we spend the rest of the year trying to make up for these losses. The membership development team largely lacks control over the one month (corporate IEEE should be able to help out here), but it can help change course throughout the rest of the year and compare how the data looks relative to other years during the course of the year.

In addition, I reiterated that if we focus solely on total membership numbers, we are going to miss everything else that is going on with membership. Getting down to the individual members that are in these aggregates at some point in the future should provide quite a bit of additional insight.

Initially, my goal was to put together a proposal to corporate IEEE to provide at least one additional data item: historical member status. We have access to historical member grade in the database, but not historical member status.This data item seemed to be the missing link that the database was not providing, but it has been admittedly difficult to determine whether or to what degree this is the case because corporate IEEE at this point in time has not yet provided data models.

Following my communication to the team of this goal, I realized that the data issues inherent in the database are likely more pervasive, because a number of other data items do not appear to be effective dated. It is an issue if I am using Oracle Business Intelligence Enterprise Edition (OBIEE) 11g to create a "new analysis", and the tool permits me to report effective dated versus non-effective dated data items.

What may be needed is an overhaul of the way the data is stored, and tasks such as this are typically not trivial. During a call to corporate IEEE last year, it was communicated to me that the database has historically had no "advanced users" such as myself, and so many of the topics that I had introduced in the conversation were simply off of their radar.

None of the presentations I have created are affected by such a data issue because they all explore data at the aggregate level, and not at the individual member level. Since the database does not yet seem to provide the ability to effectively consider historical member data in relationship to aggregate data, I am still exploring the use of other visualizations that help provide a more holistic view of the aggregate data. 

After working with simple line charts which plot member counts across targeted member grades over time, I moved to diagrams that use February as the baseline month for the year and plot percent difference in member counts from this baseline, since this is the month that typically introduces a vast majority of arrears status occurrences.

The follow-up box plots and beeswarms were intended to show monthly member count variances over the years, how these break down by member additions and deletions, and any data patterns that might exist. However, none of these visualizations show relative net difference in member counts across all years and months, so I added a variation of the scatter plot, the balloon plot (aka bubble chart) to my last two presentations, using the R language ggplot2 package.

Each data point represents a calendar month since January 2008. The position of each data point shows the relative relationship between member additions and deletions for each month, and the size of each data point shows the net difference between the two, in absolute terms (with green data points as net gains and red data points as net losses). In my first plot for January 2014 above, a quick glance communicates where the month falls relative to all other months.

The second balloon plot, for the month of February 2014, shows that it is a typical month relative to all past February months. Providing labels for every data point to indicate month would introduce significant clutter, but suffice it to say that all of the data points in red in the lower left hand corner are for the month of February across the indicated years.

Quite coincidentally, the month of March 2014 was the best month across the years covered by the database in terms of net difference. The larger the green data point the better, and the closer to the upper right hand corner, the better. While it is difficult to assess the size of the data point for this month relative to some other data points just below it, I know that the net difference for this month is larger because I revisited the source data to verify.

The Membership Development Chair requested that I walk him through the plots in my latest monthly presentation to the team (currently 21 slides), partially in preparation for his request of me to present to the executive committee next month, and I realized that this visualization in particular might benefit from some annotation for the nonanalytical.

For example, one question that was posed is why I think March 2014 was the best month when there are data points above it in the plot. So I explained that although a few months have a greater number of (positive) additions along the y-axis, these months also have a greater number of (negative) deletions along the x-axis, and the quantified net difference favors March 2014.

Outside the drilling down to individual members that will be my focus for my proposal to corporate IEEE, the only other aggregate level visualizations that I am currently considering are breakdowns of monthly additions and deletions into categories.

Member addition counts can be broken down into the following categories: arrears paid, correction addition, elections, grade transfer to, moved into, and reinstatements. And member deletion counts can be broken down as follows: arrears, correction deletion, deceased, grade transfer from, member to affiliate, moved out of, and resigned.

My next entries in this series of blog posts will likely discuss my presentation to the executive committee (which some team members are now suggesting I execute on a regular basis because of the value add that they see), the additional visualizations just mentioned, and potentially some high-level whiteboarding as to how data needs to be stored in Oracle in order for me to effectively use it.

---

Pro Bono: IEEE Chicago Section Statistician - Part 1, Part 2, Part 3, Part 5

New Book Review: "The Purpose Economy"

New book review for The Purpose Economy: How Your Desire for Impact, Personal Growth and Community is Changing the World, by Aaron Hurst, Elevate, 2014, reposted here:

The author believes that we as a society are in the early days of what he calls the "Purpose Economy", a fourth economic stage preceded by the Information Economy (a term coined in a doctoral thesis by his uncle, Marc Porat, almost 40 years ago), the Industrial Economy, and the Agrarian Economy. This new economy is centered on the needs of people to find purpose in their work and lives, a natural progression of the needs of people and the types of goods, services, and jobs desired by them. While the Information Economy still dominates, with technology largely synonymous with jobs, growth, and innovation, as Hurst began sharing his ideas with those around him, the ideas surrounding this new economic stage resonated with much of what they have been witnessing in their own lives.

As founder of Taproot Foundation, the largest nonprofit consulting firm in the United States, the author witnessed the positive social impact that pro bono work provided. His search for patterns in successful social change efforts resulted in what he called the "Five Levers for Social Change" (research, policy, public perceptions, disruptive technology, and bright spots) in his 5-part series for the "Stanford Social Innovation Review" a few years ago. As Hurst visually mapped out all of the diverse approaches to advancing progress, he came to understand that the diversity of perspectives of those around him to execute actually constitute a diversity of purpose.

The author explains the difference between "purpose" and "cause" early in the book to drive the message that cause is a noun whereas purpose is a verb. Purpose is an action. Over the course of reading thousands of applications by Taproot applicants seeking to contribute to nonprofit organizations, Hurst came to realize that people gain purpose when they grow personally, establish meaningful relationships, and are in service to something greater than themselves. But how can people be enabled to have work rich in purpose? This book explores the evolution of markets within the Purpose Economy and the key levers that can be used to advance them, the new approaches to running an organization across sectors that can thrive in this new economic stage, and the finding of professional purpose at the individual level.

After presenting examples of personal, social, and societal purpose, the author explores the evolution of our economy in greater depth, providing an analysis of the Purpose Economy (in much the same way as his uncle had for the Information Economy), and explaining that it is comprised of much more than the nonprofit sector. In subsequent chapters, Hurst presents the ten drivers of the new economy (human-scale technology, the Maslow Millennial Effect, Generation Disrupt, environmental, economic, and political turmoil, longevity, changing families and evolving roles, new social science, accelerated globalization, a shifting social context, and blending of the sectors), followed by discourses on why purpose matters, purpose myth-busting, the who, how, and why of purpose, the practice of purpose, the Purpose Economy organization and how an organization can thrive in the new economy, the purpose-driven professional, leading with purpose, and societal purpose.

As a former Taproot project manager who has also had fulfilling experiences working on pro bono initiatives both before and after Taproot, I was excited to hear about this forthcoming book. While not everyone will agree with all that the author presents, he hits the nail on the head in so many areas on the subjects of work and purpose that most will find it an engaging read. Many of my past book recommendations are limited to specific audiences, but because the subjects of work and purpose are so pervasive, an attempt to impose limits on the target audience proves difficult. In some aspects, the thoughts that Hurst shares are an evolution of those shared by other authors such as Max Weber and Rick Warren, but for a much broader readership that includes the society at large, composed of individuals with diverse backgrounds.

The author ties together topics well, and as a reader I especially appreciate the even mix of theory and practice that focuses on action, including dozens of sidebars about individuals and organizations throughout the text, and the points he makes about the purpose-driven individual and why companies seeking innovation and creativity need to think of their employees as artists. And for readers who are not familiar with the difference between "pro bono service", which uses specific skills of professionals to provide services to those unable to afford them, and "volunteerism", a great discussion on this subject matter is provided in Chapter 4 ("Purpose is What Matters"). Research has shown "that pro bono, the donation of [one's] highest strengths and talents, [is] more rewarding and pleasurable than traditional volunteerism, because it combines the elements of meaning and engagement." Well said.

Pro Bono: IEEE Chicago Section Statistician - Part 3

This post is the third in a series on my pro bono efforts outside of the workplace as Statistician for the IEEE Chicago Section, a new position that I filled this past year after the membership development team realized that ongoing data analyses and reporting was needed around declining enrollment.

The bulk of my time spent on this work over the past year was fixing existing graphs that I had inherited, moving from Microsoft Excel to the R language, maintaining existing data sets, creating new data sets, and creating formal presentations that I started distributing on a monthly basis.

After creating re-runnable R scripts earlier this year for the monthly graphs that the team came to expect, my attention turned toward creation of new visualizations that seek to better understand the data, and I began to spend more time to communicate my analyses for the nonanalytical.

My first post discussed some of my initial thoughts surrounding my early experiences on the team, and presented one of the new visualizations that I created, a heat map that breaks down top technical interest areas of members which the Membership Development Chair began using in his marketing efforts.

In my second post, I walked through some of the issues in the existing graphs that I had inherited, and showed how I fixed them in Microsoft Excel and ported them to the R language with a consistent look and feel, as well as how I began including them in IEEE Chicago Section presentations.

One of my early observations while first becoming familiar with the data is that every February membership declines drastically. After discussing with the team, I came to realize that these declines are a result of members going into arrears status after not renewing their memberships on time.

When looking at the initial graphs, one will notice that each year, total membership each January is lower than the prior year, with total membership in February following suit. One goal that the team initially verbalized is closing the membership gap with the previous year.

The problem with this goal is that because February declines are so drastic, it can be argued that the duration of each year is spent climbing back in terms of enrollment, either with new members or renewed members, and "closing the membership gap" is highly dependent on regaining previous position.

In other words, although the team is responsible for helping to increase new membership, and attempts to encourage members in arrears into renewing membership, "closing the membership gap" is not only a weighty task, but should really be seen from multiple perspectives, not just total membership.

Below are a couple examples of a new visualization that I began including in regular communication to the IEEE Chicago Section. These are based on the same data sets as the graphs I presented in my second post, but now start in February rather than January, based on a February baseline, and use a y-axis of percent difference from February. 

 

The first example above demonstrates that even though a "membership gap" exists, in terms of percentage growth from the annual February baseline, 2013 is essentially in line with growth a few years ago. Now, as I mentioned to the membership development team, one needs to be careful with this type of analyses, because the data population for the number of years being investigated is rather small, and the number of members that comprises each February baseline is smaller than the previous year, resulting in a higher percentage growth for each member added in subsequent months.

However, a cursory review of the above example also shows that even though membership is declining each year, it does not mean that percentage growth automatically increases each year. If this were the case, 2013 would be at the top of the graph, and 2008 and 2009 would be at the bottom of the graph (the year 2007 was purposefully excluded because, as explained in my last post, it is the first year of data that is available for this analyses, and it is also a partial year that does not go back to February).

The second example below is for a specific member grade, senior members, rather than total membership. The situation is the same here as well. The years 2008 and 2009 might be expected to be at the bottom of the graph, but they are not. There exists significant overlap between years as well, although it needs to be kept in mind that senior members comprise a small slice of total membership. One curiosity that was discoverded with this visualization is that senior membership percentage growth drops between December and January, and anomaly that does not occur in any of the other membership grades.

This past month, I was particularly pleased with the results of my combination box plots and beeswarms. Because the R language ggplot2 package does not provide this capability, I used the standard R language box plot along with the beeswarm package. The beeswarm package provides the ability to group individual data points with different colors, which I assigned according to the color pallette used with the ggplot2 visualizations, although the consistent look and feel was largely lost.

In my first monthly presentation that used this visualization for member additions and deletions, I included the year 2007, but data anomalies associated with the first month of data (August) that has been made available by IEEE skew these graphs, so I removed this year from subsequent iterations. In addition, the February deletions mentioned above skew these graphs each year for total members to such an extent that I now provide separate plots that include and exclude February. 

A perfect example of how removing February has a positive impact on readability are the plots of higher grade member deletions. In the first plot below, February deletions are far and above any other month, and the distribution for February across the 6 years that have been included is relatively narrow. Because of these consistent outliers, the data for the other 11 months is largely unreadable. The data points for the month of July, for example, appear to be the same across all years (note that when using a beeswarm, all data points are always plotted, and to enable visibility without data point overlap, data points are plotted next to each other). In the latter plot, the true distribution of July is made visible.

---

Pro Bono: IEEE Chicago Section Statistician - Part 1, Part 2, Part 4, Part 5

Pro Bono: IEEE Chicago Section Statistician - Part 2

This past month, I submitted a first post in a series that discusses some of my pro bono contributions as Statistician for the IEEE Chicago Section. As stated in that post, my goals are to work with the Membership Development Chair to meet his immediate needs, as well as explore some ways that we can use the data to help understand our members. In other words, look at our membership at both an aggregate and individual level. I also mentioned that in my opinion, some of the best insights will likely come from an investigation of the data at a more granular level, and then determine how findings might apply to the aggregate.

Since it was recently requested that I take ownership of some ongoing monthly reports that have been manually constructed by exporting data from canned IEEE queries in an Oracle Business Intelligence Enterprise Edition (OBIEE) 11g instance called SAMIEEE to Microsoft Excel, I am expecting the need to review at some level what has been done in the past with respect to these reports. In taking a first look at these reports, I especially wanted to gain familiarity with the graph and associated spreadsheet for year over year (YOY) monthly membership totals, because it provides a snapshot of the chapter over time, and has been used to demonstrate the annual membership lifecycle.

The first graph presented above was exported directly from this latest such Microsoft Excel workbook. My first glance at this graph immediately noticed the following: (1) every year exhibits a sharp drop from January to February, (2) data is plotted for a minimal number of years, (3) the first year is only partially plotted, and (4) not every data point is labeled. After reviewing this graph a little more closely, I also noticed two additional aspects of the graph: (5) the x-axis is labeled for year-2012, and (6) although 12 months are listed, only 11 are actually plotted.

While fixing (4), (5), and (6) for the Microsoft Excel version of this graph presented below, I also came across several other essentially trivial issues that I will not mention here. I am also not going to get into all of the ways to tweak graphs in Microsoft Excel, because the purpose of these posts are not to increase usage of the product. Regarding (2) and (3), the data used to plot the original inherited version of this graph came from an OBIEEE dashboard called "Geographic Membership Statistics", the data of which starts in August 2007.
 

An IEEE director recently explained to me, in agreement with my assumption, that the data used to populate this dashboard is different than what is available via the adhoc query area of SAMIEEE, which to my knowledge contains all IEEEE membership data for the last several decades. While a separate discussion, I have asked for additional data availability to help bridge the gap between the data available via this dashboard, and the data available via adhoc queries, in order to perform the value added analyses and visualizations for which I was initially interested as the IEEE Chicago Section Statistician.

Regarding (1), I came to realize quickly by looking at the data that a significant portion of members go into arrears during the annual membership enrollment period, dropping the number of active members in January every year. In other words, members enter a non-active status when they do not renew membership by the deadline. The difference between this status and non-active statuses is that associated members enter a grace period during which time active status can be regained without penalty. As soon as I get access to the additional requested data, I will be able to to determine which membership gains following February each year are due to new memberships, and which are due to renewed memberships, and then investigate member characteristics between these two groups.

In addition, I will be able to more fully automate the R scripts just created to graph other items in the canned reporting dashboard called "Geographic Membership Statistics". Two of the first graphs converted to the R language are presented below, both in their raw and presentation formats. The most important aspect of these R scripts is that they provide partial automation. The raw CSV data sets exported from OBIEEE are read into R as data frames, and a few (some quite lengthy) script lines later the graphs are complete with a standard look-and-feel that are void of the need to tweak like Microsoft Excel because all are created using scripts that can be re-run with updated data sets over time.
 

---

Pro Bono: Taproot Foundation - Project Manager (Draft Direction)

Pro Bono: Taproot Foundation - Project Manager (Delivery & Assessment and Closing)

Pro Bono: IEEE Chicago Section Statistician - Part 1, Part 3, Part 4, Part 5

Pro Bono: IEEE Chicago Section Statistician - Part 1

Over three years have past since my last post here on my pro bono efforts outside of the workplace. My latest pro bono contributions over the last few months have been working as Statistician for the IEEE Chicago section. The Statistician role is new for the section, and works within the Membership Development team.

Much of my time to date has been spent working to get familiar with the data that is currently available on our membership. The SAMIEEE database uses Oracle Business Intelligence Enterprise Edition (OBIEE) 11g. Unfortunately, my early efforts to obtain programmatic access to the database via IP address and port was not successful, so I create rough data sets via the web front end that OBIEE provides and then export for analyses.

At a high level, my CSV exports from OBIEE are imported to PostgreSQL, and then I use the R language with a package called sqldf to perform analyses. Traditional SQL statements can be run against database tables in this manner as though they were R language data frames, making it much easier to work with data for someone such as myself who is adept at SQL as a technical architect.

My plan is to get into more detail regarding the specifics of my setup in later posts. As part of this first post, I want to share the first visualization I created for the IEEE Chicago section a few months ago. A heat map showing the top 26 technical interest profile subjects (TIPS) of active IEEE Chicago section members, created using the above technologies along with specialized R language functions such as heatmap().

The Membership Development Chair has since used this visualization as part of his marketing efforts associated with trade shows, as part of his presentation on our membership. Creating this heat map was fairly straightforward. Although working with the R language environment and language is not intuitive for someone who specializes in Java, getting the right data to create this visualization was really the hard part.

While the TIPS list might not need an explanation, since it is essentially a list of technical interests in which members have shown interest, notice the labels for the three columns. Data was queried and sorted by primary interest, which indicates whether a member actually performs regular work in the area, followed by some work in the area, and some degree of interest, the least significant category of the three.

Traditional engineering concerns may comprise the bulk of the top 26 technical interest profile subjects (a somewhat arbitrary number that I chose, although the break off as to whether a subject made it to the heat map was set at 50 members in the regularly work category), but I found it very interesting because of my profession that Software Engineering was the top regular work subject, and one of the top subjects in the some work category. And Computer was the top subject in the interested category.

Working as Statistician for the IEEE Chicago section, my goals are to work with the Membership Development Chair to meet his immediate needs, as well as explore some ways that we can use the data to help understand our members. In other words, look at our membership at both an aggregate and individual level. In my opinion, some of the best insights will likely come from an investigation of the data at a more granular level, and then determine how findings might apply to the aggregate.

---

Pro Bono: Taproot Foundation - Project Manager (Draft Direction)

Pro Bono: Taproot Foundation - Project Manager (Delivery & Assessment and Closing)

Pro Bono: IEEE Chicago Section Statistician - Part 2, Part 3, Part 4, Part 5

New Book Review: "Data for the Public Good"

New book review for Data for the Public Good: How Data Can Help Citizens and Government, by Alex Howard, O'Reilly, 2012, reposted here:

Similar to another recently published O'Reilly Strata series text on Big Data, "Designing Great Data Products: Inside the Drivetrain Approach, a Four-step Process for Building Data Products", in the sense that it is very succinct, in this case providing focus on data for a specific purpose, rather than solely focusing on Big Data like the other series entries I have read recently, "Big Data Now: Current Perspectives from O'Reilly Radar" and "Planning for Big Data: A CIO's Handbook to the Changing Data Landscape", this quick read is expected to be a welcome addition to the reading list of anyone beginning their investigation of government data being made available to the public (otherwise known as "open data").

After discussing the case for open data and the potential that it has for public good, Howard walks the reader through several examples of how open data is currently being used, and then addresses trends demanding special attention in this space. As a consultant who has worked for both public and private sector clients, including pro bono work for another consulting firm which provides services around open data, I especially appreciate the outline of four areas that support the investment in this space that the author shares from John Tolva, the chief technology officer for Chicago, which includes trust, accountability of the work force, business building, and urban analytics, as well as the sharing of four trends that include civic network effects, hybridized data models, personal data ownership, and smart disclosure.

And the closing thought of the author speakes volumes: "The exabyte age will bring with it the twin challenges of information overload and overconsumption, both of which will require organizations of all sizes to use the emerging toolboxes for filtering, analysis and action. To create public good from public goods - the public sector data that governments collect, the private sector data that is being collected and the social data that we generate ourselves - we will need to collectively forge new compacts that honor existing laws and visionary agreements that enable the new data science to put the data to work."

Pro Bono: Taproot Foundation Project Manager - Part 1

My pro bono Taproot Foundation project team completed the Delivery, Assessment, and Closing phases Thursday night according to the Ernst & Young devised methodology (the current Project Management Institute equivalent is the Executing and Controlling phase combined with the Closing phase) executed for our nonprofit client in the Chicago Loop since April, 2009.

Back row: Sean Murray (Taproot Graphic Designer), Ben Kimpel (Taproot Web Developer), David Dubin (Taproot Marketing Manager), Michael Bird (Taproot Copy Editor/Writer), John Jiang (MCIC Technical Director). Front row: Erik Gfesser (Taproot Project Manager), Teresa Pudi (Taproot Account Director), Virginia Carlson (MCIC President, Executive Sponsor). Not pictured: Patricia Gross (MCIC Vice President). Photo by Erik Gfesser.

Metro Chicago Information Center (MCIC) is an independent, nonprofit research and consulting firm founded in 1990. The data, information and insight that MCIC provides has made them the research partner of choice for more than 1500 organizations and individuals working to improve social conditions and quality of life.

Pro Bono: Taproot Foundation Project Manager - Part 2

My pro bono Taproot Foundation project team completed the Draft Direction phase last night according to the Ernst & Young devised methodology (the current Project Management Institute equivalent is the Executing and Controlling phase) currently being executed for our nonprofit client in the Chicago Loop.

Clockwise from left: Michael Bird (Taproot Copy Editor/Writer), Patricia Gross (MCIC Vice President), Teresa Pudi (Taproot Account Director), Sean Murray (Taproot Graphic Designer), John Jiang (MCIC Technical Director). Not pictured: Erik Gfesser (Taproot Project Manager), Virginia Carlson (MCIC President, Executive Sponsor), Ben Kimpel (Taproot Web Developer), David Dubin (Taproot Marketing Manager). Photo by Erik Gfesser.

Metro Chicago Information Center (MCIC) is an independent, nonprofit research and consulting firm founded in 1990. The data, information and insight that MCIC provides has made them the research partner of choice for more than 1500 organizations and individuals working to improve social conditions and quality of life.