Hello, lovely reader! July is almost over, and I’m back to offer another quick devlog. This week has been a lot of picking through the weeds rather than sweeping advancements, so apologies if it’s a little bit boring from the outside.

First, our launch online went surprisingly well! Only one system broke upon attempting to push it to the virtual machine that runs our server–and that was a relatively easy fix. (Future note for any other devs on this project: all modules are run from the base folder of the repo, so make sure any file access your project includes presumes the cwd is that master folder, NOT your project folder!) From there, it’s been many little semantic changes made in suggestions from our advisors–units displayed more prominently and consistently when needed, updating tooltips for more intuitive descriptions, and drafting out the supplemental content that students will use as the foundation of their interaction with the module.

Another tip I can offer: Get outside eyes on you when doing anything public facing. My biggest struggle with attempting to write content for students is keeping in mind they may or may not have any experience with the central topics of our module, and thus any explanation needs to start from the absolute basis and work its way into being a more complex description. (For example: to understand neural networks, it’s important to know machine learning and to know machine learning, it’s important to know data science. Data science is built from statistics, mathematics, and domain expertise… and that’s just to understand the goal of the network–not how to build one.)

That’s been another big goal of mine, so towards that end, I’ve updated the suggestions survey embedded into our website. Anyone who has the time, I’d absolutely appreciate it if you went here to check out the page and give feedback on what’s currently there. Everything helps!

We were also lucky enough to be interviewed by Ms. Haidan Hu in preparation for an article to be written about our work! It was awesome getting to talk about the project in earnest and I really hope the article brings some publicity to our team. Educational tools really scale with how many people use them, so any exposure will help us in reaching that goal.

Beyond that, I’m happy to say the DLT is really nearing completion! The summer Expo is this week, and Dr. Rangarajan chose a representative to showcase our work in California at an educational conference. It’s been so awesome seeing this project through, and I cannot wait to see everyone else’s this Thursday.

Stay safe in that late-summer heat, and I’ll see you then!

-Maddie

Hello, lovely reader! July is almost over, and we’re nearing 75% completion of this summer program. That feels absolutely wild to write out, but I think it provides a good excuse to take a look back and reflect upon the work we’ve done so far.

Our team came in with essentially no backbone to build from, and no returning teammates to reference. Since then, we’ve taken the base of a module idea and seen it through entirely from scratch! I’ve taught myself a lot of Python in the process, and even built my own library for use on our webpage. We’ve also come a long, long way in terms of describing the project and our work, culminating in presenting to the entire campus community last week at press conferences. We’ve even made professional connections internationally, that may inspire our next projects! (No spoilers, but I’m very, very excited to see where these go.) It feels amazing to have come so far in such short order. With that, I also am SUPER proud to announce that the first draft of our module is LIVE on the web! Scan the QR below and you can see for yourself the fruits of our efforts.(PS, the image is also clickable for those on desktop.)

This version of the DLT has nearly every function we’d hoped for and more in the Bokeh visualization. Students can build their own model, watch it update, analyze and optimize all with simple adjustments. What’s still to come is in the form of our ‘supplemental’ material–many of the pieces that surround the Bokeh core and provide an explanation to the user. Our main focus as a software dev team has been, of course, getting the software up and running, but as the summer winds down, we want to ensure our module comes with all the educational bells and whistles it needs to be the best tool possible.

Looking to the future, what’s next for us? Beyond just prepping this module for its final deployment, we’re in discussion to have an article written about our work! I’m very excited about the opportunity to be featured in an educational space and hope it brings the spotlight to our little corner of the world. We’re also taking a holistic view of our team’s work up until now, and looking to implement more features for accessibility wherever possible. One of the most beautiful things about being a web-based platform is how easy it is to build in these things, unlike in traditional education. Dyslexia-friendly fonts, colorblind-friendly visualizations, alt-text and transcripts for the visually/audibly impaired, and more can all be built right into the website! These tools ensure our tool reaches the widest breadth of students possible, and gives students choice in how they wish to learn our content.

Finally, to share some more visuals and miscellaneous work I’ve been up to–I’ve been the head on designing our poster for the Mountaintop Summer Expo! (Mark your calendars for August 4th!!) There’s a fine line between visually appealing and visually overwhelming, and as a visualization team, it’s been really important to me to tread that line with caution. This poster will also be used when our work is presented later this August in California, so it covers both past and present work on the project. Take a look!

Short blog post today, but most of our work really has been preparing for this launch. I hope to see you all at the Expo soon, and will be back next week with another update!

-Maddie

Hello, lovely reader! What a week it’s been. We at STEM Visuals got to have our first proper presentation to a more public audience (other groups, educators, and visitors to Lehigh), we’ve got our module to a functional prototype stage, and it really feels like a culmination of everything we’ve done thus far. Allow me to talk a bit more in-depth about each of these things!

First, the presentation–or more accurately, a ‘press conference’. Mountaintop staff gave us all the opportunity to share our projects with the larger Lehigh community, and I was in charge of putting together the presentation content. I’m quite proud of the results, actually–feel free to view the finished product here.

These sorts of presentations seem simple but often force me to do some of the most critical problem-solving surrounding our project. Coding often is about getting knee-deep into documentation, hyperfocused on solving a very specific problem. This sort of work, on the other hand, forces me to confront more existential questions about our project–who are we catering to? Why do they care about our project? How are we stacking up with others doing similar work?

It may seem somewhat pedantic, but having these answers in the back of my mind really helps to ground our development in realistic expectations. Knowing our project, our audience, and our goals for the future allows every smaller decision to be made in pursuit of these larger goals. Not to mention, it’s just pleasant to take a break from Python. Highly recommend taking on a variety of tasks like this in the future, as having the ability to stay productive while challenging different parts of your brain can provide much-needed stimulation (or a break to the parts of your brain most prone to burnout..)

Now, this isn’t to say I spent a whole week on a presentation. We’ve also been continuing to synthesize the basis of Tyreese’s Bokeh code with my neural network library, to great success! Just yesterday, we even got to test putting our Bokeh module into the greater HTML package–and I couldn’t be more ecstatic with the results. Getting to see our work in the same format as the finished products before us was so, so exhilarating. We hope to clean up some final bugs and push this test build onto the public web by the end of next week.. but for now, enjoy a small sneak-peak!

From here, I think our work will begin to shift away from the massively technical to make a more complete, feature-rich product. We’ve been taking a strong look at accessibility features, with plans to implement screen-reader functions, colorblind-friendly site palettes, and audio/video transcription. After all, what good is an educational product if some students just can’t use it?

Overall, I feel pretty darn good about our current progress. We’re on track to finish this module by the end of August, which was our original timeline! I can’t wait to see how things unfold, and I’ll keep you updated weekly!

-Maddie

Hello, lovely readers! I hope you had a lovely Fourth of July if you celebrate, and if not, I hope you’re enjoying that midsummer weather! (Though, if you’re nearby me, perhaps I’d rather say I hope you’re staying safe amidst the smoke and storms.) We’re reaching the halfway point of the summer, and I’m so very excited to update you on my progress here at STEM Visualizations.

I left last week on a somewhat tenuous hope that I’d be able to approach the huge task of adjusting my neural network code to be more modular and user-customizable: fancy buzzwords to mean that more things needed to be able to change on a dime. That was my very first task of the week–generally, the process would mean isolating a single task in the creation of a neural network, identifying its’ inputs and outputs, and encapsulating that task into a recognizable function. Here’s one of the simpler examples:

When we begin to train a neural network, an important factor we want the user to be able to adjust is how much of our dataset the model may use while training, how much to be used to validate results, and how much to hold out for final testing. This process is central to most data-science processes, and we’re no different. So, as I said: what comes in and out of this particular function? In goes our dataset, already split into X (inputs, sometimes called features) and Y (outputs, or labels). We also need to know how we’re splitting this dataset–that means we’ll expect three percentage values to represent each split. Let’s call those test-, train- and val-Split respectively. What comes out of this function? Well, each of the splits for both X and Y, which we’ll expect to be proportionate slices of the original X and Y data frames. We’ll also do something a little sneaky: once the dataset is split, we’ll ‘pop’ the Time column off of the data frame and save it separately. This feature won’t help in training the neural network, but it will help us to plot our results. We’ll take those as outputs too, for a total of 9.

Now that I’ve explained it all in English, here it is in native Python:

The point of this exercise was partially to force myself to read through my own code, cleaning it up whenever necessary and adding comments for future legibility. Further, it makes later calls to build the neural network far cleaner and more intuitive–no more copy-pasting large blocks of code to do the same actions, just a few function calls that point in and out to the correct locations. I repeated this process for each piece of the neural network puzzle until I had a full library that could train and save a whole neural network with one function call! (Let me tell you, seeing that the final function worked as intended was satisfying on another level.)

With a modular neural network ready to go and able to accept a myriad of different training parameters, I was able to move on to another looming question: what training parameters were best for our problem? After all, this is one of the main questions we hope students will try to answer with our module, so we really ought to know the answer. This process was not nearly as fun as building a function library, however: in the end, it boiled down to running the network under thousands of conditions, saving the best one (as decided by minimizing validation loss) as we iterated. I won’t share the ‘answer’ just yet, in case a future student finds this page. Suffice it to say, I was slightly disappointed to find I was already quite close to ideal parameters before spending multiple hours on this optimization loop. Computer Science can be a fickle field, let me tell you.

Finally, there was one more issue I faced at the end of last week: remember my comparisons between expected and model-driven predictions? At best, you could call my model… clumsy. Our training data draws relatively smooth curves and straight lines, but the model struggled to follow that path, often dipping unexpectedly or losing course of reality. I’m glad to say that in retooling the training parameters, this issue was fixed as well! Here’s how our model predictions shape up to expectations now:

I sincerely promise this does not depict the same graph twice with different titles; that’s just how accurate this model has gotten. I cannot overstate how happy I am to see this model handle the regression task so well and feel confident that it’ll prove invaluable when implemented into our module.

It’s been one heck of a ride this last month or so, and I’m happy to have reached such a milestone. With much of the backend coding complete, I can move on to other ways to help our project shine–including prepping for next week’s Mountaintop Summer Experience Press Conference and designing supplemental material for the final web module.

Hello, reader! The month is almost over, and with it comes time for another update post. In case you aren’t caught up, I’m Maddie, working with the STEM Visuals team at Lehigh University. Our team works on making web-based modules for the classroom, helping to teach complex STEM topics in a more interactive way. Our current project is working with Lehigh’s Dr. Menicucci to create a “Digital Lab Twin,” where students can build and use their own machine learning models as a part of their experimental process. Last week, I was able to show off a simple neural network fitted to our data–this week, we turn our focus towards applying that model, and ensuring it’s efficacy in solving the problem we want it to.

I’d mentioned the first thing I wanted to show here was my model making multi-step predictions. See, the model is currently trained to make single-point predictions–it takes the experiment at a time ‘t’, and returns the experiment at time ‘t+1’. It’s gotten extremely accurate with those single-point predictions, but all alone, those aren’t extremely helpful to a researcher. What is more useful is the ability of the network to feed it’s output back onto itself, generating a series of predictions over time. With that, we can emulate the experiment growing and changing, and see how our target variables shift!

That sounds complex, but this was actually the simplest problem I got to solve this week. We load up our model in evaluation mode first, then prep an empty dataset ready to hold our predictions. The model reads in data one row at a time, starting with our given initial conditions (remember, this is what the user will change over time!), and predicts the next time step, loading it into that empty dataset. Then, we simply access the next row of our output data, using that as input to predict another timestep! Rinse and repeat until the experiment is ‘over’ (read: the empty dataset is full) and you have a whole batch of model-driven predictions.

We can plot those much the same way as we did our single-step predictions! For clarity’s sake, here’s a quick comparison of what that looks like.

..if you aren’t impressed, don’t worry–neither was I. Here we run into a key issue that faces a lot of regression problems like our own. Trying to feed outputs into inputs often leads to straight-up wonky predictions, because any error early on can seriously debilitate the model’s accuracy as it goes on, compounding into a whole mess in a short matter of time. There are multiple avenues of fixing this issue, which has been one of my main projects this week and likely continuing into the next. Ideally, I’ll find a ‘solution’–a combination of model parameters which can provide the accuracy in predictions we’re looking for, so that when used in the classroom, Dr. Menicucci has a way of guiding students towards a model that will help with their research (if they don’t find a better one themselves!).

I’ve only just begun digging into said avenues of model improvement–there are many. We can fine-tune the model’s ‘hyperparameters’, a very fancy word to describe changing how the model learns. We can adjust the loss function to a variety of choices, changing how the model evaluates accuracy, or shift it’s learning rate, how much the model adjusts itself with each new batch of learned data. Even more broadly, we can adjust the ‘prediction horizon’, attempting to push the model to predict further in the future than a single timestep. This is the ‘science’ in data science, and definitely one of the most interesting tasks I’ve been given on this project.

Outside of my little bubble, we now have more clarity on what our finished project will look like! The module has two main ‘sections’, one for model editing, and one for prediction. Students first get to play with the model build itself, changing all the things I wrote about above and more. They can tease about with the system, and get a hands on view of what it’s like to try and train a model to learn a complex equation like this one. Eventually, once they’re ready to experiment, they’ll be given our pre-made ‘solution’ to the edit module, and moved towards the prediction side. Here, they can try and optimize the photobioreactor system, adjusting experimental parameters at will.

..needless to say, this scope is far grander than what we had in mind at first. Really, just about doubled! However, much of the work is already done–we’ll just need to modularize each piece of the puzzle that is building a machine learning model, so students can pick and choose which pieces to fit together.

“Just” is a relative term, as that is a big project to tackle. I look forward to it, and can’t wait to update you soon!

-Maddie

Happy Friday, dear reader! Another week down, and another chance to reflect and update here. In case you don’t feel like reading the past entries, let me catch you up to speed:

I’m Maddie, one of 4 students working on the STEM Visuals team at Lehigh University’s Mountaintop Summer Experience program. Our current focus is developing a “Digital Lab Twin” Module for use in a lab here at Lehigh, in tandem with our advisors Prof. Rangarajan, Dr. Menicucci, and Dr. Thiagarajan. The module hopes to use machine learning to model the photobioreactors Dr. Menicucci’s class will build through their labs, allowing students to play with a virtual sandbox and find interesting parameters to experiment with physically. The project is web-based and open-sourced–all past modules can be found here.

Now, away from the elevator pitch and onto this week’s progress! We’ve pulled together as a team quite nicely if I may say, and this marks the first week of what I’d call truly tangible results.

My week started by solving the problem that’d really been stressing me out for the last few weeks: getting our synthetic dataset ready to go. No machine learning is happening without that data, so it was a prerequisite to even the most simple of testing. If there’s anything to take from us, it’s this: whatever you think will be difficult will likely be simpler than you believed. What you believed to be simple, however, will likely end up with you feeling like this:

Seriously. I’d hoped this would take maybe a week at most–and here we are, finishing it at the start of week 3. All things said and done though, it’s alright–the time spent researching ML technologies definitely made implementation far, far easier. Once I had my data, it was a matter of a day’s work to transport it into the test model I already built (for the nitty-gritty on that process, see last week’s post!) Once again, Python comes in clutch with a great community and plentiful documentation, so all that I really had to do was pin down what my problem was in order to find a working solution.

Now, this is a visualization project, so I’d be remiss if I didn’t share the output of the graph by this model just like last week. As before, first is a loss graph, showing how accurate our predictions were against the number of epochs, or runs through the entire training set of data.

That sharp decrease shows that our model is learning quickly! The fact our MSE reached zero is something to possibly improve on, as it may be a sign of overfitting. However, for a first attempt, it’s promising to see!

Next, here’s one of the actual output models. The X axis represents time; the 150 hours that the training experiments were run for. The Y axis shows concentrations of Lutein, a wanted bioproduct synthesized from a photobioreactor. All 100 experiments are overlaid atop each other, creating the spread of blue and pink dots you see–blue was data used to train the model, and pink is holdout points to test later. Green shows what the model thinks the Lutein concentration should be, given the conditions of the experiment prior!

Isn’t it pretty? Getting a cool-looking model like this is uber-satisfying, and part of the fun of a project like this. Our predictions aren’t too far off, either; that’s a relief! However, this isn’t the final plan for this model.

See, what’s happening up above is the model predicting only one time step ahead of reality. It’s a shallow neural network in that sense–each set of inputs is tied to one set of outputs, representing that next timestep. Ideally, our model can be run multiple times, feeding its own output back into itself to generate an entirely model-made set of data over time. Beyond administrative work for the team, the rest of my week has been focusing on creating this exact idea–a visual of what our model attempts to predict over longer periods of time. Sadly, I’m not quite finished with it as of writing this post–but expect that to be the first thing I gush about next week!

As for other strides forward this week, we were challenged to put together a presentation surrounding end-users and really hone in on the audience for our module. Alex designed this new logo for it, and it’s really cute!

Perhaps someday we’ll find a more catchy team name than simply “STEM Visualization”, but that day isn’t today. At least our logo’s eyecatching now!

Further, allow me to share my favorite visual from the presentation I designed myself:

These presentations offer a welcome change of pace from focusing solely on code, and creating bright visuals for them has become one of my favorite weekly tasks. I’m excited to make more as we move on!

Looking forward, it’s all about optimization on my end. I want to adjust the neural network to be more robust, and that starts with measuring how accurate it already is. Further, we’ll want to explore other models, so once the neural net is finished, I’ll likely switch to helping my teammates who are currently attempting a similar task to my own with a Gaussian Process model. It’s been a heck of a week, all the better for having been four days!

Here’s to hoping progress remains this smooth. See you all next week!

-Maddie

Hello, dear readers! It’s amazing how fast time flies–we’re already 1/5 of the way through this summer project if you can believe it. With our final team member Alex back in Bethlehem, it really feels like we can hit the ground running. We’ve had our hands full, and I’m very excited to update you on my progress!

Our week started off much as last week ended–exploring different Python libraries and collecting needed resources to use later in the project. I personally spent some time with PyTorch, another open-source library for building Machine Learning models in Python. The absolutely amazing web resource Learn PyTorch has great documentation walking through the process of building and testing models for different scenarios, and I cannot recommend them enough as a starting point for others interested in ML. Following their lead, along with guidance from Prof. R., I was able to build a sparse feed-forward Neural Network in the system!

I’ll explain the basic structure of the network, and then show off the results I was able to get.

So, for the layman, let’s lay down a foundation. What’s a neural network, why am I making one, and how did I do it? First, perhaps most important: neural networks chain together layers of computation called ‘neurons’, that train by reading through large datasets and slowly ‘learning’ what makes that data tick. The network adjusts its decision-making parameters as it trains until eventually, it can make decisions related to whatever you taught it. Most networks focus on one of two major tasks; regression and classification. The former attempts to predict numerical values whereas the latter attempts to identify an input according to a list of possible outputs. Our project focuses on the former–ideally, experimental data goes in, and the model predicts experiment results without ever having to touch a lab.

Before we get that far, though, we need to start at the absolute basics. My first given task was to predict one value, using a simple function as training data. I won’t take you through the nitty gritty for this example, since my code heavily mimics the tutorials above. At a high level, the process is simple! Data needs to be generated–in this case, I had Python generate the first-order rate equation often used in chemistry, at Prof R’s request. This data gets loaded into PyTorch tensors and split into a training and testing segment. (More complex models will include a third split as validation, to finetune model parameters before full testing.)

We then define the model network; in this case, it’s a Sequential line of fully-connected linear layers (ones where every neuron is connected to every other neuron) along with activation layers to signal when the neuron before has found a critical value. In my case, this turned out like the nearby snippet of code. This model follows a ‘pyramid’ structure for its neurons, starting with the 1 input, expanding it outward, then slowly bringing it back down until the model has 1 distinct answer. You can also see two further variables, a loss function, and an optimizer. The former tells the tool what measure to use to read accuracy, and the latter is how the model chooses to update neurons during training.

We then train this model, feeding it into the network many, many times–otherwise known as epochs. Epochs are broken down into ‘batches’ of data, so the model can optimize more than once per dataset cycle. Batches are pushed forward through the model, and then the optimizer performs a ‘backward pass’, counting changes in neuron weights. The optimizer takes a run-through, updating according to the algorithm we specified earlier. This process can repeat as often as we, the designer, want, but it’s best to stop the model when the loss can’t drop any further. This helps prevent overfitting, a phenomenon where models learn the exact dataset rather than the underlying concept they represent, and can’t generate good predictions on unseen examples.

Here’s where we see how we did! Above is a ‘loss graph’, which in its’ simplest terms shows us how accurate a model is as it continues to learn from the data. We’ve plotted our loss rate, in this case, mean squared error, against epochs, how long the model has been learning.

Finally, we get to see a model make predictions! This network was trained on data from a first-order rate equation, a common function used in chemistry. This actually shows a rate of change–one that’s relatively linear in nature, though not quite. While this particular model isn’t spot on, it’s surprisingly close for such a small set of training data and a simple network structure.

..whew, that is a LOT for a ‘simple’ model! However, this exercise helped a ton in learning the process of generating any machine learning model and gave me fun graphs to show off in the process (always a bonus). To anyone else attempting a similar project, I seriously recommend this method of starting small and working your way up to the project you have in mind. Incremental progress lets you test your own knowledge as you go, and mistakes won’t feel nearly as punishing as they happen.

Our team also had some really great discussions about further project goals, especially centering around what problem we hope to solve for Dr. Menicucci’s laboratory class. Even this early on, I consider the end-user experience probably the most crucial aspect of development. Our success is measured by whether our models see use in a classroom, after all! Thus, the module now has a name: The Digital Laboratory Twin. All in all, I feel really great about our progress, and can’t wait to see where development takes us!

-Maddie

References:
Predictive modeling and multiobjective optimization of diamond turning process of single-crystal silicon using RSM and desirability function approach – Scientific Figure on ResearchGate. Available from: https://www.researchgate.net/figure/Typical-artificial-neural-network-structure-64_fig2_333206147 [accessed 16 Jun, 2023]

https://www.learnpytorch.io/

Hello again, prospective reader! It’s great to see you again. We’ve had quite the week here, so allow me to catch you up to speed, starting with the basics. In case you missed my introduction, I’m Madeline, one of four student members of the Stem Visualization team for Lehigh University’s Mountaintop Summer Experience. (Whew, what a long title! I’ll refer to us (me, Madeline, along with Tyreese and Alex as full-time Fellows and Alicia as a part-time Associate) as the team, and the program as Mountaintop for simplicity’s sake.) We’re lucky enough to have joined this ongoing effort to create interactive web models for education, focusing on allowing students to interact with complex STEM topics. The project website can be found here, as taking screenshots doesn’t really do it justice. Past models have included infectious disease modeling, zero energy cooling systems, and reaction kinetics; this year we’ve pivoted towards models surrounding bioengineering and data science. Further, I’ll update this post to share the blogs of my teammates as I get them!

With that said, I ought to get to the purpose of this posting: sharing progress with you all! It’s been a journey this week, that’s for sure. The first and most intimidating challenge of the project was simply learning the makeup of our predecessor’s work–installing prerequisite programs, gaining access to the Github repository, and studying the code to understand how the finished products came to be. This is in part made challenging by the fact it’s an all-new student team this year. Hopefully, blogs like this will help anyone coming after us to make this learning process a little less hectic.

Once we prepped our digital workspaces, the next hurdle was finding places to dive into this year’s module. Our grand concept is to present to the user an interface that models reactions within a photobioreactor. These setups grow microalgae under specific conditions, with the goal being to maximize the synthesis of a wanted product.

The specifics of bioengineering are not my strong suit, and I won’t pretend that they are–for further reading, linked here is the paper serving as the genesis of much of this project. What matters for us is to know that this is a complicated process with many variables at play; each of which needs to be fine-tuned if product synthesis is to be maximized. This is a goal that’s greatest suited to the technology I’ve spent the week studying, so allow me to walk you through the basic idea!

Machine Learning has a lot of hype burning around it right now–so much so that the words alone lose meaning. To be clear, we aren’t attempting to build AI in the sense that we’re training a computer to speak, hear or see. Our model, in comparison, is actually somewhat simple: Given a certain set of initial parameters and a trained history of what the reaction should look like under those conditions, return a prediction of what the reaction looks like in the future. Thankfully, this fits the definition of a bread-and-butter data science task: regression. With the task defined, we can move toward deciding what type of model to train.

Regression models come in many shapes, from simple linear predictors to Gaussian function predictors to multi-layer neural networks. If none of those words mean a thing to you, you now understand the place we were in at the beginning of the week–and the gargantuan task ahead of us. Much of this week’s progress came in the form of tons of research just to understand the principles behind each possible process. After all, there’s no way to be certain about the efficacy of a model if you have no clue what it’s doing! To that end, I spent about two days compiling basic information on different regression methods. You can see the fruits of my efforts below, and if any of them intrigues you, there’ll be more links to the sources I’ve referenced.

At this point, reader, your head is likely swimming–as was mine. Let’s do a lightning round then, and demystify each of these concepts. First: Logistic Regression, perhaps the simplest process, and the backbone of many others. We read in a set of inputs in an attempt to predict a single binary output (it’s either true or it isn’t!) Ironically, despite having regression in the name, this is actually a classification model. We can change it to become a probability predictor, though–that’s what the equation is for. The efficacy here is measured through a Receiver Operating Characteristics curve, or ROC. By plotting the area beneath that curve (AUC for short), we get a value that approaches 1, with higher values representing a more efficient model!

These aren’t enough for our problem, though. We aren’t predicting something binary, but rather we want to see values over time. That’s where the latter two models come in! These models can be used for classification and regression, and thus are a bit more complicated. Once again, I’m hardly an expert–our job is simply to understand the tools in our belt so as to apply them properly. GPs are probability-based, non-parametric predictors: That is to say, instead of taking simple inputs to get an output, the GP attempts to align the data across possible functions it could represent and presents you with the probabilities that generates. Once again, a fascinating system, but not quite aligned with our needs. That brings us to the last and certainly not least!

Neural networks are probably the most complex of the systems I’ve looked into in terms of finding a fit for this project. Just the basics took me a whole whiteboard and a few hours of reading–and I’ve only just started covering the surface. But at their most basic, here’s the idea: We create neurons, units of processing that adjust values according to a parameter weight and bias, normalize that generated value, and pass them onward. We create column-like layers of neurons and stack them together, chaining the decision-making of each layer together to create a more complex decision process. The inputs and outputs are visible, but each layer in between is generally hidden–all of this to simulate a more humanlike function of learning. Neural networks themselves come in many forms, utilizing different algorithms to tweak the decision weights, measure loss, and improve over time. We’re even in luck, that a specific type of neural network is suited perfectly to our issue of photobioreactor optimization!

That brings me to the final project of the week, getting a working demo of one of these models fired up. The theory is great, and nothing gets built without at least a surface-level knowledge of the underlying system. However, I thought it’d be prudent to study a functional example–and more importantly, to get it working on my own hardware. Much of my time was spent studying this demo code from the Keras/Tensorflow module of Python; libraries specifically built to aid in creating neural networks. It walks through the entire process of generating a simple Recurrent Neural Network (RNN), one designed around a very similar problem to our own: predicting future temperatures based on past weather conditions. Kudos to the Keras devs for such excellent documentation!

That brings us to today, where we stand as of the end of week one. I’m happy to say we’re feeling confident in our progress thus far! The project’s been broken into key components, and each of us has an area of expertise to put our time towards. Next week, our final teammate will be back from traveling, and we can get into the nitty-gritty of not only implementing a working model but ensuring its educational value for the Bioengineering concept at hand. Thanks for reading this far, and have a great day!

-Maddie

Further Reading:
[1] Victor Zhou – Intro to Neural Networks

[2] Victor Zhou – Keras For Beginners

[3] Katherine Bailey – Gaussian Processes for Dummies

[4] Arthur Mello – Logistic Regression: The Basics