Office 365 and Machine Learning idea won the ESPC17 Hackathon
The idea, Office 365 as a better collaboration space
The initial idea came from @Joon Du Rant and we added on the table few fundamental questions including:
How we can give the users information they need to successfully use Office 365, therefore collaborate better?
Can we create Office 365 communities that can help and learn from one another?
Office 365 users can learn from others with a similar usage profile, we just need to identify people with similar interests and add them to a group. Sounds easy, but many organizations fail at that for different reasons. There is more, to improve the collaboration and knowledge sharing we don't just add bunch of people in a group. We also have to identify the collaboration champions, users who are willing to help the communities i.e. SMEs and experienced buddies to moderate, share and spread information. With Office 365 we get enough data to start investigating user behavior.
Ok, we have the data generated by Office 365, but what to do with it?
Use machine learning and the Office 365 logs to identify groups of people with similar interests
The Office 365 logs can be used in different ways ... auditing, troubleshooting, but why not also for learning about the user behavior and interests?
The plan was to use Azure machine learning on the data from the Office 365 logs to identify clusters of people with similar behavior. Joon enabled detailed logging on our DEV server so he can have enough data to play with. The main part of the Azure machine learning, models used and results is Joon's work and he covers that in his blog Using Azure Machine Learning to understand user communities on Office 365 while I will focus on integrating the Office 365 products to complete the flow.
Integration between Office 365 multiple products and Azure ML to complete the flow
The complete flow explained in short:
- Get data from the Office 365 logs (the red cloud icon is the start point)
- Move it into Azure machine learning
- Build clusters of people with similar interests
- Move the clusters report data to SharePoint list
- Attach PowerBi reporting and dashboard to the SharePoint list
- Attach Microsoft Forms to the clusters with users and send them Microsoft Forms survey by sending the invites from Microsoft Flow
- Analyze user's feedback and the clusters data together
- Identify groups with similar interests and champions collaborators
- Crate Yammer groups based on Office 365 Groups and invite the users by the help of Microsoft Flow again
- Repeat the process after some time to reorder the groups based on the user's all the time changing need of information
Machine learning is not enough, we have to identify moderators willing to help
The machine learning will give us clusters of users, but it is very hard to predict who is willing to help the others and we are not experienced enough data analysts to make that assumption from the ML results only so we created additional survey to ask our users. Based on their answers we would try to find Office 365 collaboration champions (power users) willing to help the others and share their expertise and add them as group moderators so they have high rank in the groups.
Microsoft Forms, Microsoft Flow and SharePoint to get user's feedback so we can identify possible group leaders
We created Microsoft Forms survey and added it to a SharePoint page so the users can access it from SharePoint.
We also created Team Flow so we can send invites to the users in the clusters.
We asked the following questions to identify our users:
What do you use Office 365 for?
How confident are you in your use of SharePoint?
How confident are you in your use of Microsoft OneDrive?
How confident are you in your use of Microsoft Microsoft Teams?
How confident are you in your use of Microsoft Flow?
How confident are you in your use of Microsoft PowerApps?
How confident are you in your use of The Microsoft Office suite?
Do you help other people to use Office 365?
How often do you find it difficult to perform the task that you wanted in Office 365?
Which of the following sources do you use to find out how to perform tasks in Office 365?
Where very important is what they answer on the `Do you help other people to use Office 365?` part so we have rough idea if they a candidate for a moderator.
It was fun playing with Microsoft Forms because the survey mode also uses machine learning to predict your answers so you do not have to type them.
Note: We did not have time to survey real users so for the purpose of the hackathon the survey was filled with random test data automated via PowerShell script.
Yammer and Office 365 Groups as collaboration and knowledge sharing platform
The final part was to invite the users to Yammer groups so they can collaborate effectively. Now, when new Yammer group is created, it uses Microsoft Office 365 Groups which make it easy to manage and automate. Yammer is also very good collaboration tool for masses of users and this is what we needed because we plan to implement that for a fairly big enterprise organization and the groups would possibly be thousands of people. Based on that we created Microsoft Flow to invite people in Yammer-Office 365 groups based on our analysis from the clusters and survey data.
There is the final result, we have the first users invited to the collaboration champions Yammer group :)
SharePoint site, Microsoft Flow, and Power Bi as hub for data analysis and reports
We turned SharePoint Communication site into analysis and reporting hub for our project so
- The aggregated clusters data from Azure Machine Learning was uploaded in SharePoint list by the help of Excel and a little bit of PowerShell.
- The user survey data has been added to a SharePoint list by the help of Microsoft Flow creating new record in the list once a survey is completed.
- The lists were data source for Power Bi Dashboard with the following data visualizations "The machine learning clusters", "Survey data - Cluster source data", "Survey data - User Confidence", "Survey data - User Profile."
- We used SharePoint modern pages and Power Bi web part to bring back the Power Bi charts and dashboards into SharePoint.
Microsoft Flow to automate and integrate different services
Serverless, no code solutions like Microsoft Flow can speed up the implementation of a process. We knew that and we took advantage of it. Our idea was to integrate as many as possible Office 365 products, but with no code and with few team Flows we were able to do the magic.
Conclusion
Well that sounds hard to achieve for two-three days hackathon and it is! We had to unitize our time, experience and tooling to be 100% productive. The good part is that Joon is super experienced and organized professional, he is solution architect with more than 10 years of experience and myself, I worked with some of the best Office 365 names like Radi Atanassov and Vasil Michev for many years. The success turning point was also that we decided to do it with almost no code, but just integrating different Azure, Office 365 products to move and transform data, but it all worked.
This is solution that we are about to implement in our organization and will try the improve user's life. The modern era is here and we have all those complex tools provided for us by the cloud so why not we use them to bring better experience to our users :).