61: Polly Update


  • Today I took Polly, our pug, to the neuro-specialist to find out what’s going wrong with her back end.
  • The vet hypothesised 1 of 3 things. 1) Disk problems, 2) Cyst (SAD) or 3) Constrictive myelopathy.
  • It turns out after the MRI that she has a little from columns 2 and 3.
  • Her immediate problems are caused by a cyst, but she has malformation in the spine which causes the cyat to form.
  • So we’ve been given a few choices. 1) do nothing, prognosis, complete immobility and double incontinence in 6-12 months. 2) steroids, which will relieve the symptoms but not resolve the problem 3) surgery,
  • Lucky we’re insured so we opted for the surgery.
  • I’m anxious for her, I have all the usual worries about the surgery, but I’m also nervous about her being at the hospital for the week. Will she sleep well etc.
  • So, to whoever is reading these posts. Send some good thoughts.

Reading: The case for working with your hands by Matthew B Crawford. (64% Complete)

Wordle: 572 5/6

60: Sunday code day:


  • This morning my good friend Craig popped over this morning.
  • Before the Pandemic, he used to pop over every Saturday or Sunday, and we’d chew the fat and usually end up doing some coding together, but it’s been about 3 years since we did that… until today.
  • In the week, Craig and I were discussing how most systems have terrible reporting.
  • They usually have some high-level stats that doesn’t really help you to answer the real questions you have.
  • So today, we took one of the reports from his college MIS system and tried to see if we could generate some more appropriate insights.
  • While working on the data, we found some pretty cool things in Pandas.
  • I will write a post with all the tips and tricks I need.

Reading: The case for working with your hands by Matthew B Crawford. (32% Complete)

Wordle: 572 5/6

59: Saturday


  • Really quiet day
  • Tam and I went to B&M for a few bits. Then I walked banjo with Craig.
  • Evening was spent watching telly and doing a new jigsaw.

Reading: The case for working with your hands by Matthew B Crawford. (32% Complete)

Wordle: 572 5/6

58: SharePoint (Graph API)


  • We are making good progress on the Sharepoint migration script.
  • We’ve now got a script that will find all the site IDs, which will allow me to use the Move API to organise the files and folders.
  • The next steps are to create a spreadsheet that will make it easy for me to map the source and destination folders.
  • I also need to run a test to ensure we don’t overwrite any files in the source destination.
  • As this will allow me to run a ‘delta’ migration.
  • I also need to add some defensive code for any API call that might fail. So I can log the output and retry.
  • I’m still not happy with the auth, but now I’ve got the refresh token working I don’t have to do the access token dance with the browser which is simply annoying.
  • The script should be finished this weekend.
  • So I’ll do a full breakdown as soon as I can.
  • I’m creating this script outside of Jupiter notebook which is an interesting

Reading: The case for working with your hands by Matthew B Crawford. (32% Complete)

Wordle: 572 5/6

57: Polly, Sharepoint


  • Today was a mixed bag.
  • I took Polly to the orthopaedic vet to see if we could find our definitively what’s wrong with her back end.
  • She’s still having difficulty walking and some incontinence, and the painkillers don’t seem to be helping at all.
  • The vet seems to think it’s not cruciate ligament disease as was previously though. Instead she thinks that polls has compression in her spinal cord which is stopping the never from working properly.
  • With this is mind, we’ve been referred to a specialist for an MRI on Monday.
  • If it turns our that she does have a spinal issue they she’ll be given steroids which should bring immediate relief of her symptoms.
  • So fingers crossed we can get that sorted out and start next week on a positive note.

  • Aside from that Adam and I teamed on to see if we could crack this data migration project.
  • You may recall that last year is was battling with Microsofts Sharepoint Migration tool (SPMT) to move files from Egnyte to Sharepoint.
  • After a lot of to and fro with the tech support team at Egnyte I manages to get this resolved by increasing the throttle on the number of API calls allowed per second.
  • ✅ we can now migrate the files as is from Egnyte to Sharepoint
  • Next step is to re organise the files in SharePoint to the new folder structure.
  • I had hope to be able to do a proper folder mapping in SPMT but thus proved unsuccessful.
  • I then looked at doing the manually but the share number of folders/files made this unsustainable.
  • So I resorted to writing a script using the Graph API to move the folders.
  • Connecting to Graph API with python appears to be very fiddly, with an awful lot of poor documentation.
  • I planned to create Azure App that had application-level scope to do this, but I can’t find any document to support getting an AccessToken without requiring generating a url and looking at the return address for the token (I.e. User-based authentication).
  • The good news is that between Adam and I, we managed to get the building blocks of the script together.
  • Tomorrow’s job is to finalise the script and then run it against the test migration to see if it’ll do the trick.
  • When I’m done, I’m going to document it here so that I can reference it later.

Reading: The case for working with your hands by Matthew B Crawford.

Wordle: 571 5/6

57: Pandas – Filtering Records


Today I had one of those, aha! moments with Pandas which has made my life a lot simpler.

Here’s the story.

I have an excel file which contains a list of discount codes, some have been used, and some have not. I need to go through the list of codes and find out the following:

  • Have we assigned a code to anyone who has now left the business?

To find out who the ‘leavers’ were, I passed a list of employees and a list of used codes to a function and then iterated over the list of codes, putting any employee who is not in the used code list into a levers list

def FindLeaversWithCodes(people, used_codes) :
    #Find people we've sent a code to who are no longer in the HC Report
    _leavers = []

    #Get the Employee IDs of all the people we've sent codes to.
    _people = used_codes['personNumber'].tolist()
    #loop through each employee id
    for person in _people:
        #if this employee id is in teh current HC report then save as Current Employee
        if person not in people['Person Number'].values: #if this employee id is NOT in the current HC report then save as Leaver
    return _leavers

However, because I move things from a Dataframe to a list, I then had to filter the original Dataframe by the list of employee numbers, _leavers in the list to generate a new XSLX file in the original format. Incredibly inefficient.

    #Filter the used codes list to show the leavers
    df = used[used['personNumber'].isin(Leavers)]
    df.to_excel('../leavers.xlsx', index=False)
except Exception as error:

That first line of code df = used[used['personNumber'].isin(Leavers)] should have given me a clue as to what the FindLeaversWithCodes function should actually be doing.

Rather than convert to a list and use a for…in loop, I should have simply filtered the Dataframe to get the records I need.

Now the code is much simpler

def LeaversWithCodes(people, used_codes) :
    #This person has been assigned a code
    #This person is not an employee
    return used_codes[~used_codes["personNumber"].isin(people["Person Number"].values.tolist())]

the ~ character before the used_codes["personNumber"] is a ‘not’ symbol. I’m doing the same as before but this time filtering the original Dataframe where the personNumber is not in the People Dataframe.

Wordle: 570 4/6

56: Representing knowledge for AI


  • I started the second lecture from the HarvardX course CS50’s Introduction to Artificial Intelligence with Python.
  • This time we’re learning about AI can help with understanding knowledge.
  • I wish I had paid attention to the logic classes I had when I was a kid.
  • I’m seriously going to have to concentrate on this one. The content is not specifically hard. It’s just a lot to internalise.
  • It’s been around 25 years since I did any formal learning (obviously, I’ve learnt from books and training courses since), so getting used to lectures has bee interesting.
  • I’ll write more when I’ve finished the video.

Wordle: 570 4/6

55: Digital Employee Experience.


  • Today I started pulling together my thoughts on the ‘Employee Graph’ I want to integrate into our Employee Experience Platform.
  • The purpose of our Employee Experience platform is to apply a UX layer over the top of an organisation’s employee digital tools.
  • Alongside our more substantial products:
  • Intranet
  • Recognition
  • Performance Review
  • Rewards (Discount manager)
  • We also integrate into existing platforms such as Oracle HCM, and Microsoft 365.
  • However, the unique aspect of our solution is that we can aggregate the data from all these platforms and turn it into insights.
  • I want to make sure the insights are beneficial to the user (performance reviews, feedback), the line manager (team management), and the organisation (workforce planning)
  • So now I’m working through what this reporting dashboard could look like and the value it will return.

Wordle: 569 6/6

54: Lazy Sunday, HarvardX: introduction to AI.


  • After yesterday’s fun, and not getting home until gone 01.00, we had a very quiet day today.
  • Our sole focus for the day was a new 1000-piece jigsaw puzzle, which, sadly, we didn’t finish by the time we stopped at 22.30.
  • Apart from that, I finished the first lecture of HarvardX’s CS50 introduction to artificial intelligence with Python.
  • The course is one of Harvard’s free lecture series released under the HarvardX brand and is, so far, a great introduction to the fundamentals of AI.
  • The first lecture is on search, where we learn about search problems.
  • We were introduced to several different search algorithms
  • Uninformed search is a general-purpose search algorithm which operates in a brute force way as it does not have additional information about the state other than how to traverse the tree (so it is also called blind search).
  • Depth-first Search. The search algorithm will follow the nodes to the deepest possible depth before evaluating the next available path.
  • Breath-first Search: The search algorithm will look at all child nodes before progress to the next level of the tree.
  • Informed search is a search algorithm that knows its end state and uses that information to estimate how close it is to its goal.
  • We learn about greedy best-first search and A* search.
  • Then we were introduced to adversarial search problems (think Tic-tac-toe), where we were introduced to the following algorithm
  • Minmax – we calculate all possible moves and assign a value. Player 1 will try to minimise their score to win, and player 2 will try to maximise thier score to win.
  • Alpha-beta pruning is where we calculate enough of the subsequent moves to see what has the highest or lowest score (depending on if you are minimising or maximising) before making a move.
  • Depth-limited Minmax where (in more complex games) we only calculate our next moves up to a limited depth (10 moves) and assign a probability that a move will be winning.
  • It is a well-put-together lecture, as you’d expect, and the content was new to me, which was awesome.
  • Next, we’ll learn how to extract knowledge from a body of text.

  • Reading: Agency by William Gibson (100% Complete)
  • Wordle: 568 0/6

54: Stranger Things Experience


  • For Christmas, Ellie bought us tickets to The Stranger things experience in London.
  • So, once again, due to the Train Strikes, I found myself driving into central London.
  • First stop, I dropped off some things for Ellie, then we got the tube Brent Cross and walked to the show.
  • First, let me start by saying that, like most of the western world, I things Stranger Things is a masterpiece.
  • It’s produced one of the best scenes in television history. You know which one I mean, don’t you.
  • So, to say I was excited was an understatement.
  • The experience is split into two.
  • Firstly you participate in some immersive theatre.
  • Invited to Hawkins Lab to participate in a sleep study, you are welcome to the lab by some cranky scientists, all slightly unhinged.
  • Then you are briefed by a member of the PR team, and everything goes downhill from there!
  • I won’t ruin the surprise, but it was an enjoyable experience, both Ellie and I had fun, and there was indeed some draw-dropping moment.
  • Also, the action was led by the main series actors, which was pretty sweet.
  • Once the main show is over, you visit some of the locations from the series.
  • The setting reminded me of an exhibition setup, with a stall for each element.
  • It was pretty much a glorified gift shop with ice cream from Scoops Ahoy, Pizza from Surfer boy Pizza as well as merchandise to buy.
  • But that didn’t distract from the overall experience.
  • There were plenty of photo ops too which was fun.
  • All in all Ellie and I had a great time. I’ll give it a solid 4 out of 5.

  • Reading: Agency by William Gibson (100% Complete)
  • Wordle: 568 0/6