K-12 leaders and teachers want quicker insights and alerts from their learning data silos – delivered to them
Schools create volumes of diverse data in school management, cloud and learning systems every day. Across multiple online systems and manual collection efforts, learning data grows at an exponential rate. Most times, key learning data is never unified. Sadly, this is the Greek tragedy of data in schools. The real hero, learning evidence, capable of helping every teacher personalise student growth and learning, is rendered powerless by its inability to be unified. The catastrophe? Teachers must work harder to understand each student’s learning need without unified assistance from existing data sources.
SIS, LMS and formative assessment platforms collect hundreds of thousands of learning data transactions. NAPLAN , PAT and ALLWELL contribute many more thousands of point-in-time learning records. Cloud delivered systems like O365, Google and Mathletics collect volumes of learning data hourly. Clearly data supply is not the issue, consolidated data delivery and visibility is. Here’s the hint; this is the main causal factor of the tragedy.
Right now, consolidated data delivery and visibility is still a big challenge for schools. How do schools leverage all the learning data they have ‘everywhere’ into a clever ‘thought-support’ capability for teachers? Can technology unify multiple data sources to augment insights and instincts? Can this information find leaders and teachers to answer their unique questions? The answer to all of these questions is yes, when you approach the challenge the right way.
It’s critical that school leaders think about data strategies as a journey rather than a single destination or one off project. No school ‘does analytics’ in one pass and if they do, they only have one, often disappointing, pass at it.
Implementing an active data strategy that lets teachers find opportunities and issues quicker, collect stronger data and then share all data, needs leadership support and guidance. You really can’t expect to move from ad hoc data use to automated data analytics in one pass.
Using learning data to quickly identify issues, understand trends and measure success is what teachers and leaders want. The framework to execute and support this must combine three important ingredients; organic interest, teacher PD and a technical ability to deliver. We believe delivering new initiatives the ‘last mile’ is the biggest risk that every ‘system’ initiative faces.
How do you deliver an integrated data capability and culture the last mile?
The most effective risk mitigation strategies focus on internal adoption, inclusion in the process and efficacy, all more important than a cackle of data consultants.
Start by being tactical. We can recommend six points schools should consider when thinking about creating or extending an environment to start a data culture that uses data and evidence to support teaching and learning.
- Insist on agile and economic capabilities in preference to expensive bespoke analytic projects. When disparate data sources come together, it often takes a bit of time to learn where the relationships are and then where the true value is. Iterations need to be quick and economic to run and re-run. When your future state starts with a huge budget, the future can be fragile.
If you over-engineer what you think you need, you will be disappointed. If you find yourself writing detailed specifications, hit reset. Remember the data itself and the BI tool does not create any real IP. In this data capability context, the IP (intellectual property) is created from engagement, culture and efficacy around using data to improve teaching and learning.
Always expect that 90% of the time, you will continue to rely on your cognitive intelligence to rationalise data stories and findings.
- Understand the dimensions, importance and weighting that should be applied to each of the data sources you have. Too much fuss is often made over obvious data, simply because it’s simple to access and easy to understand. Have a good think, learn from others. You can duplicate without having to replicate. You don’t want people off piste with half-truths.
- Think about how you could collect more granular data through better processes and how to bring this data into the mix. If you don’t collect data at the right level of granularity, the data is of limited use. Do you have gaps in your data collection processes?
- Ensure data insights and alerts find your leaders and teachers in their context, not the reverse. Gaining insights into data should not become a new burden or need further academic qualification. When learning data is easy to use, teachers use it.
- Deliver a simple teacher-driven ability to personalise inquiry and alert scenarios. Avoid building one size fits all chart strategies. Everyone must chant ‘data must come to me’. Save everyone from being an over or underwhelmed user.
- Don’t get caught up re-working and transforming data, you will never stop once you start. Work and build data ‘as it is’ letting the system support the range of data diversity. Focus your time thinking about what you want to know leaving your data platform to cope with diversity.
If you are looking for a platform to transform your data silos into a single source of learning evidence, take a look at Literatu Learning Ledger and watch the simple explanatory video.
The last-mile of learning ‘analytics’ delivery.
Teachers make the biggest impact on learning when they have the right information, all the time. It’s 2018 last time I looked and this is a simple idea. Having an ‘all the time’ capability of access and insight across all teaching and learning data is what every educator wants, needs, deserves…. What are the options?
Teaching and learning data has become a critical input into school improvement programs. Knowing where more effective and efficient programs can be implemented needs clarity and speed of evidence to support teacher instincts. Getting a flexible and ‘current’ data capability into the hands of teachers remains atop of the most wanted and still elusive list for many schools. Whoops, I should have included the concept of economy as well.
Some schools have structured some of their school data into BI platforms. From what we see and hear, many of these schools are yet to deliver these ‘dashboard’ systems the last mile, that is right into the hands of every teacher in their specific teaching and learning context. Delivering the last mile is complete when teacher adoption and ownership drives the initiative. Delivering the last-mile to teachers is what we all need to focus on as well as the cost of doing so. By the way, BI is an acronym we use in a generic context rather than in reference to a product.
Many schools aligned to BI platforms have linked admin data like SIS systems that span many areas, even scheduled maintenance. As one Principal recounted, ” we have managed to build an expensive rear-view mirror that our drivers are not looking into. They actually wanted a new heads-up display.”
For people that don’t necessarily understand the ‘learning data’ conversation, they should think about how they learn and realise that every place where there’s a feedback loop likely presents opportunities to apply automated learning technology to collect and surface meaningful data. Save the innovation thought for teaching because there are now recombinant platforms that allow teachers to collect and surface meaningful data to support teaching. Literatu Learning Ledger is a great example.
Megabytes of meaningful and current learning data is found in dozens of surrounding systems like Canvas, Mathletics, Flourishing, Mathspace, Edrolo, Essential Assessment and the routine PAT series. Most of this data is not unified into a single view or BI initiative. We have found over 30 learning and pastoral sources of data in schools, many critical if you ever wanted to look at a whole-of-student view, that simply don’t make it onto the BI radar. All of these integrations into BI would be expensive and laborious to maintain. Add a new data source and the BI data team starts again.
Personally, I think much of what BI is doing out there in schools is akin to what Powerpoint does in meetings. It makes a few points, has nice UI transitions and charts and pretty much sends everyone to sleep when the room is dim. Many teachers are conservationists at heart, saving their energy for the next big idea every time they hear the word ‘analytics’. Teachers know data is important to inform teaching and learning, they just don’t see pre-configured charts helping them improve student learning in their daily context. We do acknowledge that there is a layer of every school’s team that loves BI charts, we get that too.
Delivering data ‘the last mile’ means getting every teacher involved, cracking the ‘keep it simple’ mantra around access and personalising insights for each teacher. Value to the teacher must emerge through a whole-of-student view. The ‘last mile’ of delivery is where data has to really work for each educator.
The step up from BI, is AI where data works much harder and smarter. Do teachers see the average grade chart, (after they have just done the grading), or do they get an alert identifying where the learning gap is widening for specific skills? I think the later is their need and preference. These are the big challenges we are working hard to solve without introducing another thing to do or another system to learn. No one needs that. Teachers ask us for data that comes to them. That’s the difference we see as the one worth making.
We are on a mission to give each educator a unified whole-of-student view of engagement, performance, learning and growth in the simplest and most engaging way possible. Welcome to Learning Ledger. Ledger is alive and well and set to go for every school. We already have great data integrations that turn on with Ledger and a heap of opportunities to open more data sources along with personalised student and parent learning ledger views.
Join the Evolution!
“If data is not about improving learning and teaching in a class or student context, why are teachers looking at it?”
Mark Stanley Sept 12, 2018
I recently read a great McKinsey article that had clear and simple advice for companies building data analytics platforms. I thought it was a great template to align the article from a big business context to a practical K-12 school context, morphing the same 5 steps into a game plan for building a great data analytics platform in schools. Thank you McKinsey!
The wholesome messages around building a data platform are the same; focus on a problem, take incremental steps and take people with you on the journey. In many ways, the data journey schools embark on and follow actually defines the destination they arrive at. There is no big bang data analytics process that anyone can or should recommend for schools.
If you don’t have internal data expertise, you really have to work with a partner who can take your team on the data journey, coaching your team on the types of questions they could ask and how they can explore data. Don’t buy a BI solution in a box. Look for cost effective, extensible platforms that are designed for use by teachers and leaders, rather than IT people.
Step 1 – Make sure that everything you do delivers an impact measured in months, not multiple months or Terms.
In 5 seconds – Aim to make either a rapid return on your investment or fail fast. The best way to do this is to pick an opportunity to use data that will make the biggest impact in the shortest time. Can you access data that is complete and come up with a way to leverage the information contained within, quickly?
In developing and prioritizing use cases for your data and analytics platform, the single most important criterion is speed to impact. Schools that obsess over value from the start and constantly push for high-impact, quick wins are those that do best at winning over skeptics and keeping up the momentum of change.
Many analytics platform programs share a common problem: value is only seen at the end, after all the pieces are in place. Don’t wait for long term big wins, you have to win quickly or fail fast, learning from each result.
Step 2 – Use the data you have to build in bitesize progressions
In 5 seconds – Build data capabilities one data source or segment at a time. Additional data should build incremental value and clarity piece by piece.
Two common obstacles hold schools back from building data and analytics capabilities.
The first obstacle is the analysis paralysis that sets in when schools over think what it’s going to take to achieve something. In our experience, there is no need to start with a big bang “ mindset. Success comes from building capabilities piece by piece and connecting each step you take directly to teachers who will benefit the most from the outcomes
The second obstacle is the perceived need for a massive amount of data and ETL ( extraction, transformation and loading) time before any meaningful insights can be created. Beware the ETL / BI consultants. School data is in voluminous supply, yet relatively simple in structure.
Many schools just don’t realise just how much good data they have in an analysis ready-state. Others just get stressed about what it is going to take to mine the data they have. There is vast amounts of school data collected by third party assessment products. These platforms should supply structured data downloads that can quickly be consumed and added to existing analytic capabilities.
Quite often schools assume their data will require thorough cleaning before it can be used. The most common data conundrum in schools is what we call nom-de-plume syndrome. This is when one students is known across multiple systems with a different name per system. These are really just silly annoyances that make combining data in schools more frustrating than it should be. This does not qualify as expensive data cleansing and when these issues are found, they need to be fixed in source systems.
Step 3 – Use data analytics to solve real teaching and learning problems
In 5 Seconds– Aim to solve a problem, within a Term. The believability index will lift, people will grow in confidence and the momentum you make will continue.
Don’t lose sight of the problem you are trying to solve. Don’t let data people create another complex data silo. The data team must work with the people who have the problem, to solve the problem.
Three things to remember :
- Use your platform capability to augment and improve on something that you are doing now or trying to do now. At least people will be invested in the challenge if the challenge is underway. EG. Looking at diagnostic data to find skill gaps that are impacting on current learning in Maths.
- Make sure you embed your new data capabilities into existing processes. There are teachers out there every day. What about getting alongside what they need and making an impact there. No-one needs more systems to learn. Teachers want data to come to them rather than look through a pile of charts.
- Help people to adapt to new capabilities to find an opportunity for impact. Be it an ah-ha moment, clarity on what is happening or directing more targeted teaching to where it is needed, impact is the target outcome.
Step 4 – Invest $2 in professional development and support for every $1 spent creating data analytic capabilities.
In 5 Seconds :– Ensure you focus on people and processes, not shiny new tools. No matter how advanced the tool, it will be worthless without the talent and structures needed for managing and maximising its impact. Charts are really overrated.
Introducing data capabilities is an investment in cultural change and so “seeing how it goes” is the wrong packaging and message from the start.
It really is important not to roll out a shiny new toy without some good use case and early achievement wins under your belt. Take small steps, experiment with quick turnaround cycles and work in a context with a group of teachers and leaders. You have to understand what it takes to make a difference. Under promise and over deliver is the key to success.
The more agile you make the process, the more believable it will be. Make sure there are people who can articulate why you are doing this and the benefits they find.
Importantly , you must nurture internal champions and build data into a PD context where teachers are enhancing their impact and effectiveness using data.
Step 5 – Democratise data access to continue innovation from within
In 5 Seconds – Gear your platform to democratizing data. Make data available to all your employees, teach them how to use the data, and see what unsuspected value they can unlock for your school.
- Crowdsource ideas across your team – work with teacher hunches as use cases for analytics discoveries
- Seal data sources as stable and ready. As you build in new sources of data, certify where they are, the quality and get on with it
- All data should be visible by everyone. Self service tools need to be flexible per user rather than structured to the point of useless
- New ideas need to be quickly implemented and shared – if the use cases get stuck, the usefulness will die. Let teams suggest improvements and get them done quickly , keep data flowing to the frontline of the school where it can make the biggest impact on teaching and learning
- Wins need to be shared and celebrated
Not that long ago, there was a choice to be made – Start a project offline – kind of a big buck and bang process or do something that was more agile. Terms ‘skunk-works’ came into play because it was covert usually and could be trashed at any time.
Neither of these approaches are viable anymore. If schools want to get going with data, and have a result , take note of the 5 steps that make the biggest difference to success. The key to success is to win quickly or fail fast.
The ‘D’ word is back with more focus and attention. DATA is the word and it is kick-starting new conversations about how to target student success – again. Unifying data to show the ‘whole of student view’ and to support personalised learning is a must have for schools moving forwards. It seems the ‘need to deliver stronger student outcomes’ reset button was pressed hard by Gonski 2.0.
While banks and business have been executing KYC (know your customer) strategies and analytics for years, schools are starting to rethink what they could do with their DATA. Could schools really apply similar KYC strategies, possibly modifying the acronym to ‘KYS’ and use data to better ‘know your student’? Many school experiences with data analytics have delivered expensive rear vision mirrors. Schools know they need to look forwards into the ‘now’ of student learning, be responsive today and better manage what happens next. Using current DATA is a great way to support agile teaching opportunities.
In Australia, Gonski 2.0 re-surfaced a reasonable goal that the education system should deliver a year of learning for every year of school attendance. I would suggest that to do that effectively for each student, schools must focus on a ‘KYS’ strategy built on the premise of a ‘now’ capability. With advances in thinking around data and learning records it’s now a good time to take another look at what schools can do to get a data culture and ‘now’ capability up and running without starting a big project. We suggest a strategy that starts at D and works up to C, B and then A.
Schools around Australia are about to be flooded with more diagnostic data as we write this blog. The annual NAPLAN gates are about to open. What if this and other data could quickly open learning insights into students for teachers. Is this not the future of information that we want?
ABC and D (for Data) ABC has always been a simple metaphor for describing a three-step success process. It’s also the simple acronym that has guided, and oversimplified, the data approach taken by some schools. Access some data (usually obvious data), Build charts with some BI talent and Corral support from people to look at what we found. Easy as ABC? Not really. There is limited value in all of this for teachers and growing learning. If we were to suggest a more complete approach to building up the components of a data strategy, we would:
- stick with the ABC metaphor. It’s known as simple and that must remain.
- add a ‘D’ making it a 4-step process
- change the meaning of each step
- start the process with D and work your way to A. Everything starts with data.
Creating the ability to store and access appropriate levels of data is a pivotal foundation of this discussion. It’s a fact that schools have many systems collecting information in well-defined data silos. SIS and LMS systems are just two systems that have established feature boundaries. Many other systems collect data and add significant value, but all stand alone. The storage capability needed to deliver a whole-of-student view also needs some re-thinking. Aggregating disparate data in various shapes is a prerequisite to everything. To be economical and effective, all data must remain in the authentic shape and form in which it was recorded. Replication of full source system data sets is not a viable option. Subsets of source data, pointers and summary facts that represent the target data, need identification and consolidation.
Storage systems able to understand all types of data formats with the ability to bring everything together on demand will be the backbone of the system. Good data partners will have this capability. Schools must be offered agility of data, a standards-based approach to working with data and opportunities to integrate directly from their own data stores. When this is in place, the real excitement begins.
D is for DATA integration
There is so much data available in schools. Data supply is not the problem, simple data integration is. The key role of data integration is to combine multiple data sources to provide users with a unified view. Working under the assumption that all data is good data, schools usually have over 20 contributing and disparate data sources brimming with information. Have a look at just some of the data sources we work with here.
The common link in all learning data is a student reference. If you have 20 different sources, you may have up to 10 different student reference points. This stops the whole-of-student-view in its tracks. This is a key issue that needs attention in any data initiative. Then there is data dimensionality for each data source. Systems that measure mastery have different data dimensions than assessment systems that record percentage scores. Some data links to a curriculum, other data sources link skills. The point to make is that every data source has a contribution to make to a whole-of-student view and each data source should be loaded natively without loss of any dimensions.
The main objective of data integration is to deliver an immediate factual result to teachers. As soon as data is loaded, teachers should have immediate access in the context of their class and for each student. As more data is available, contexts broaden and so does the power of inquiry. Teachers should be encouraged to bring their hordes of data forward for inclusion. Don’t forget to include as much pastoral and wellness data as possible. The relationship between wellness and scores is significant. Teachers are right, it’s actually not all about scores. Achieving a quick turnaround on data visibility and access is key to progressing to C – Connectivity.
C is for Connectivity
Connecting to data is the next sustainability challenge of every system working with data. When you have worked out what data you have, where it comes from and what it contains, you really need to build in some automated connectivity to make data loads come to life – hands free where possible. Data comes from everywhere, much of it is stored in 3rd party systems and available only in simple download formats. The objective remains, bring as much data into one core supply system as possible. No-one gets a picture of a student by shopping across several disconnected data sources. These simple data formats, sometimes complex, need to be dealt with quickly and economically. There is no sense investing in expensive data connectors when data comes to you only once or twice a year. Working with good partners should see these data types fully supported so that you get the connectivity to data that you need, quickly.
New data sources should not break the bank either. If any data process requires extensive human intervention, the chances are it could get too hard and the outcome won’t be impactful. The best outcome is always for all data to be current, updated and reliable. Economy and turnaround are the two key drivers. Some LMS systems and SIS systems can support direct access to data with some schools having a level of direct access to their databases. If this is you, again, partners you work with should have connectors and options for you to leverage. When you have good connection and visibility of your data, start exploring step B – Building Alerts.
B is for Building Insights and Alerts
Building Insights and Alerts – also goes by the name of Business Intelligence. Once you have your data loaded, accessible and visible in your context, the next step is to bring Data Alerts into view. We use the term Alerts because in an ideal system, the data should come to you. Shopping for ideas and alerts in an online data mall of charts and dials can get overly complex to navigate. Try to resist building tickly and complex BI interfaces for teachers. There is no real intelligence contained in plots of hundreds of data points for teachers who are looking for specific alerts or clear threshold information.
Also resist the temptation to dumb all scores down to be an average. The cute animated dial that says ‘Average 56%’ will miss outliers and hot spots every time. Alerts should be personalised for leaders and teachers, by leaders and teachers. The classic teacher hunch needs to be supported with flexible rather than fixed inquiry modes. Data attributes and options should be available for inquiry without teachers knowing what is necessarily available. ‘Following your gut’ is a true cognitive skill rather than looking at a chart we prepared earlier. Alerts should scale across all data sets allowing pastoral data to be included as an influence alongside gradebook data or summative results.
Once data has been loaded and connected, the system should support the ‘hunch’ – in any context, across school , class or student data. Alerts should show movement, showing the same results compared to last term, year or week. If an alert is tracking a hunch, you want to see movement without having to track or back-track the detail yourself. Building out Alerts and insights should not be not a technical or expensive task. Teachers and leaders need a simple system that supports them using data to improve teaching and learning. Literatu CEO Mark Stanley is right when he says , ‘when learning data is easy to use, teachers use it’.
A is for Assistive and Adaptive intelligence
With data volumes available, wouldn’t it be good to have some assistive insights into it all? It’s quite easy for technology companies to over talk AI, what’s possible and what it can ‘do’. As a comparative to human intelligence, AI still has a long way to go. All artificial intelligence is initiated and structured by humans. Computers have no general intelligence that does the cognitive processing we do every day. Recent research pitched the smartest AI machines at the cognitive level of a 5-year-old human, with the exception that a 5-year-old child knows the difference between right and wrong and can pick the red hat up off the floor.
So, what can AI deliver now. Well the answer is DATA crunching and a good level of “what we have noticed”. With over 20 data sources for a school of 1000 students, each with say 200 learning ‘observations’ per year tracked across 5 subjects, you are up to a quantum of 20,000,000 learning events your school has stored. Welcome to AI and what can be done.
Did you know there is a strong correlation between Music in Year 7 and mathematical ability? Did you know that English skills are on the rise and have been in Year 8 for two consecutive terms? AI is a viable opportunity that schools should not miss. It needs three things to make it work. D for data (lots of it), C for connectivity to keep supply happening and BI only to get a sense of direction as to what you think is a valid hypothesis.
AI is a process that sets structured and unstructured sequences of discovery in motion. The most common starting point for AI is correlation discovery and that is something that can be done without taking out a second mortgage. You do need to have a good supply of data! You will love it when a panel opens in your day and says, ” I thought you might be interested in this”. You will be amazed when you see what computing power can do. It’s just a bit quicker than us because that’s all it does, right?
The core intention of discussing these steps, strategies and challenges in some detail is inspire schools to get motivated about data and what it can do. Sure there are challenges but these challenges all dissipate when you have the the right platform, strategy sequence and data partners. DCBA can be as quick or methodical as you like, it is actually all about you. If you start with D, you will accelerate to B and that will be an awesome start.
Don’t get fixated on building visuals into data. All that will do is display the explainable, and restate what is already known…really. BI is not charts. BC is Business Charts, the I stands for intelligence. They are quite different and after some weeks of looking at charts, your teachers will go back to their cognitive powers of perception and human intelligence. Technology should be working with you on your hunches and finding the gold in the hills that opens a whole new opportunity. This is what the mission should be.
This week the future of ATAR in Australia was discussed. It seems the ATAR has lost its significance and magnetism for employers, seemingly Universities also. Imagine being able to hand a student a complete learning ledger at the end of their schooling years, a digital ledger of achievement across all subjects and participation, certified by the school. In the near future the focus will move away from exams towards a journey record of lifelong education and progress.
This record must be supported by longitudinal data. It’s amazing how fast this will happen but it won’t happen without data and data integrity. It all starts with schools. There really is minimal competitive advantage in collecting learning data just so you can say you have it and you use it. Every school could implement the same or similar systems, every school has access to the the same or similar data sets. Not every school has the leadership and commitment to sponsor a proactive data culture to build data collection, let the data speak , listen to what the data is saying… and then act on those messages. Yet again cognitive humans are the only ones who can build and realise the competitive advantage contained in data.