This one’s for Vanessa, who pointed me at this article. The claim is that a major part of web traffic (up to some 69%!) comes from social sharing that happens outside the well-known networks like facebook, google+, or twitter, and that this behaviour actually predates what we now call “the social web”. But how do we measure it and what can analytics contribute to a better understanding of this “dark social”?

First thing I would point out is that we need to make a difference between the social web and people being social with or without technology. The early web was not a social space as we know it today. Yet, we used non-browser-based applications to communicate and share links by e-mail, netmeeting, Messenger or ICQ, as well as chatrooms. And we still do! A great deal! These do not (yet) show in referral analytics and hence they can be called personal, direct, or “dark”, which mostly means person-to-person. It’s a digital word of mouth propagation as opposed to broadcasting or spamcasting that happens in social and other sites. Technically, however, there is no reason for skype or any other desktop application to not provide analytics information, and with the majority of folks now on a webmail service, it seems merely a matter of time and implementation.

Historically speaking, the “Web” wasn’t social in the early days as its applications and protocols were distinguished more decidedly from say ftp, gopher, smtp, and so forth. Dedicated channels were used to map our social needs onto. Thanks to an explosion of webapps and cloud services, in today’s sloppy jargon “web” is used for everything that “connects”.

What’s different with the social web today is the transparency and interconnectedness with which link sharing, web traffic and social activities happen. Instead of personal messaging, many people send links to groups of people (friends), even when they know it’s only relevant to a few. “Likes” and +1′s integrate the dissemination of re-shares much better and, most of all, easier.

When looking at my own web stats, which I pondered over in another posting, what strikes me is the amount of spidering that goes on. This is the real “dark social”, since it makes my posts and links go places they’ve never been before (including some rather dodgy sites). This now seems to account for more traffic volume than RSS subscriptions. Integrated analytics that captures the actual consumption of my content, be it on my own site, on aggregators, or on pinterest/paperli-type pages is still lacking. For all I know, one person liked my stuff and re-shared it with an audience I can only guess at.

Finally, there is much more technical integration than we usually acknowledge. Thanks to notification services, I actually accessed the said article, to which Vanessa referred me, from my e-mail, not from the social network where she posted it. So it would be wrong to assume that all social sharing that falls into the dark social space was unrelated to social networks.

In summary, it is probably more astounding that social networks within only a few years account for nearly a third of web referrals. In this, they’ve clearly run a dent into the importance of e-mail and other traditional channels in the dissemination of stuff. Maybe it is precisely this notion of the traditional ways of sharing becoming legacy behaviour, which hitherto stopped better analytics in this area. However, in the interconnected web we live in, content-based rather than address-based analytics would seem to be the way forward, including aggregated data of third-party consumption at remote re-share sites.

Day three of the Edusprint on Analytics kind of returned to day one, taking about the enterprise side of data. A lot of this was repeating what had been said by the representatives from UMBC. The topic on how large amounts of data were managed in HEIs, showed, this time by example of the University of Washington and Arizona State University, how important leadership and top-down governance is for learning analytics.

Institutions react to the challenge of learning analytics as they usually do – by establishing committees! It was therefore no surprise to hear that data governance in UW and ASU was committee driven. They (the committees) in turn spawned a number of task forces, who, I suppose, did the actual job. I am left to guessing how effective this mode of operation really is, but it seems to be the only way universities know. At that the UW approach sounded very heavy handed and formal, presented like the independence of the United States which I do find a little over-exaggerated in approach and self-perception. However, the key aspects of their institutional approach to data are of importance to note:

- consistency in approach to data

- clear models, definitions and processes

Especially clear definitions and consistency of data policies according to defined needs and roles was a point strongly emphasised. Standardisation of data formats and policies (e.g. access rights) make things internally definitely a lot easier. They reduce the need for data cleansing  and guarantee equal access to information across departments and units.

But the presentations also had some omissions. Maybe this is due to these things being of more indirect nature, but no-one talked about the ethical and privacy procedures, if and how they were set up. I consider ethics a fundamental principle of data driven information systems. And the rights of a data subject (an individual student or teacher) to investigate what information about them is made available and how it is used is simply a matter of transparency and fairness. After all, analytics provide support for decision making, including student grading. Therefore, a solid investigation and complaints procedure is as important as creating the information in the first place.

Day 2 of the Edusprint on Analytics was devoted to the impact analytics can have on learning and teaching. Marsha Lovett from Carnegie Mellon presented an interesting approach to derive meaningful inferences from analytics about students’ learning states.

To start with in a quick poll, participants (of whom most declared themselves IT professionals) showed a clear preference toward intuitive judgements being made for measuring student performance and progress. 59% of some 300 spontaneous respondents said they evaluate how their course is going by the “feel”.

Carnegie Mellon have attempted to move beyond this in interesting ways, and with consideration of cognitive learning theories. Their base assumption is that learning is skills specific and that students draw on their skills to carry out instructional activities. On this they built a quantitative model of skilled learning, by also assuming that frequent practice leads to better skills. Through this, one can get better and deeper insights into the learning state of a student than by just looking at raw performance data, such as test scores or attendance. Unfortunately, Marsha did not explain how the digital actions of students (e.g. clicking on a resource) are translated into these skills, but still, the thought model looked pretty convincing.

When a student does some digital learning, the interactions are analysed and computed into an inferred student state, which, if I understand correctly, identifies the learning skills in use, and compares them to an expected state. This is presented to the student and instructor in a dashboard. The dashboard displays key aspects of the student’s learning state and also gives some recommendations.

What I liked about the presentation was the mention of analytics needing to be actionable. This is close to my thoughts encapsulated in the Learning Analytics framework. It leads to the question: are the dashboard displays of learning skills intelligible to students and teachers in determining their next action?

The general idea that I see represented here is similar to what some of my colleagues work on, that is using dashboard displays as reflection amplifiers that make people consider their position and perhaps change direction or efforts. Another thing that is reasonably clear is that comparative info about similar students performance gives more room to reflect than just one’s own data.

An important point raised by the audience was in how much learning analytics takes web clicks as proxies for learning. The response to this is that we need to look beyond simple clicking behaviour. Students perform linked actions and we need to perceive them contextually as patterns of learning and action sequences.

According to Educause, 24% of HEIs have Analytics as a major priority. Even more see it as an important development, and no-one perceives it as becoming less important in the future. This leads to more than 90% of institutions being at least very interested in analytics.

As with most things, money is the key driver to introduce analytics in institutions. The hope is that with better information, costs of e.g. drop-outs etc. can be reduced. At the same time, affordability is seen as the biggest obstacle for analytics. Admittedly, it surprised me that this response in the Educause survey put concerns about privacy and data misuse into second place. In my opinion, this may reflect the managerial thinking of respondents from university management. Be that as it may, collaboration between IT and institutional research are seen as most important for progress, but have to be supported by the leadership.

We have heard about UMBC’s strategic approach and institutional commitment which led to a cultural change in information management of the institution as a whole. The key message from Educause, therefore, is to position analytics as a major institutional investment.

Interesting talks on the first day of the Educause Analytics Sprint with some 750 participants in the online webinar. Three senior managers from the University of Maryland, Baltimore County (UMBC) explained their motives for extensive use of analytics in their institution. The focus, in my vocabulary, was on business analytics and not learning analytics, although learning is of course the major business of a university. Still, the educause definition of learning analytics goes like this:

"the use of data and models to predict student progress and performance, and the ability to act on that information."

You can already see how this fits well into institutional performance indicators and reporting activities. Nevertheless, it was highly refreshing to hear the UMBC president speak with great enthusiasm about how analytics has changed the culture of the institution. He mentioned how every student retaking a Chemistry exam costs the university hundreds of Dollars, and that analytic real-time observation and in-time intervention can bring these costs down. In some cases, courses were redesigned due to data information gathered. It was also mentioned that the new "openness" of courses led to more collaboration between departments sharing innovative ways of delivery.

From the institutional perspective, UMBC presented the argument that information gathering and reporting wasn't only for some administrators in the central office, but a general matter of the entire institution. This cultural change led to more transparency and exchange. Rolling out reporting to members of academic staff led to more empowerment and has transformed the institutional information culture on the rest of the campus. The president stressed that through analytics his institution became more agile in information management.

An interesting trust issue that came up from the president was to engage institutional researchers (IR in American) in analytics, since this is where most of the analytic expertise lies. Analytics is not only asking about numbers he explained. To answer how many students graduate is not as important as to ask why they graduate and what makes them successful. Similarly, analysing from which schools their students mostly come, enabled UMBC to direct their recruitment missions more concretely to these schools rather than dividing their attention randomly.

Although the argumentation was very convincing, all this can still be interpreted as UMBC blowing their own trumpet well. But, I perceived a more subtle message that may be more relevant than the showcases they mentioned. This message is about strategic vision and institutional commitment. I know of not many Vice-Chancellors or University Rektors who would even attend such an event on analytics, not to mention making a case for it. The establishment of data quality teams that deal with data cleanliness, security and access, shows institutional commitment right at the top layer of the university. Through this, they eliminated lengthy cleaning processes of databases and produced higher data quality, accuracy and standardisation than ever before. Compare this to some institutions that are unable to even say how many staff and students they have.