Data Science & Machine Learning Articles

Flask Data Science Machine Learning

Create a Complete Machine learning Web Application Using React And Flask Create a complete Machine learning web application using React and FlaskPhoto by Alvaro Reyes on UnsplashI have always wanted to develop a complete Machine learning application where I would have a UI to feed in some inputs and the Machine learning model to predict on those values. Last week, I did just that. In the process, I created an easy to use template in React and Flask, that anyone can modify to create their own application in a matter of minutes.Highlights of the project:The front-end is developed in React and would include a single page with a form to submit the input valuesThe back-end is developed in Flask which exposes prediction endpoints to predict using a trained classifier and send the result back to the front-end for easy consumptionThe GitHub repo is below. Fork the project and create your own application today!https://github.com/kb22/ML-React-App-TemplateTemplateReactReact is a JavaScript library created by Facebook to help make working with User interfaces simple and easy to develop and use. It’s one of the leading languages for front-end development. You can read about it here. The best resource to learn about React is its documentation itself which is very comprehensive and easy to grasp.Flask and Flask-RESTPlusFlask and Flask-RESTPlus allow us to define a service in Python which will have endpoints which we can call from the UI. You can learn more about developing a Flask app from my article below.https://towardsdatascience.com/working-with-apis-using-flask-flask-restplus-and-swagger-ui-7cf447deda7fDescriptionI used create-react-app to create a basic React app to begin with. Next, I loaded bootstrap which allows us to create responsive websites for each screen size. I updated the App.js file to add a form with dropdowns and Predict and Reset Prediction buttons. I added each form property to state and on pressing the Predict button, I send the data to the Flask backend. I also updated the App.css file to add style to the page.Template viewThe Flask app has a POST endpoint /prediction. It accepts the input values as a json, converts it into an array and returns to the UI. In actual application, we’ll use the same data to make prediction using the classifier stored in classifier.joblib and return the prediction.Prediction displayed on UIReset Prediction will remove the prediction from the UI.Starting the templateClone the repo to your computer and go inside it and open two terminals here.Preparing the UIIn the first terminal, go inside the ui folder using cd ui. Make sure you are using the node version 10.4.1. Once inside the folder, run the command yarn install to install all dependencies.To run the UI on server, we will use serve. We will begin by installing the serve globally, post which, we’ll build our application and then finally run the UI using serve on port 3000.npm install -g serve npm run build serve -s build -l 3000You can now go to localhost:3000 to see that the UI is up and running. But it won’t interact with the Flask service which is still not up. So, let’s do that.UIPreparing the serviceOn the second terminal, move inside the service folder using cd service. We begin by creating a virtual environment using virtualenv and Python 3. You can read about virtualenv here. We will then install all the required dependencies using pip after activating the environment. Finally, we’ll run the Flask app.virtualenv -p Python3 . source bin/activate pip install -r requirements.txt FLASK_APP=app.py flask runThis will start up the service on 127.0.0.1:5000.ServiceVoila! The complete application will now be working properly. Yaay!!Using the template for own use caseTo understand the process of using the template for any model, I’ll use the iris dataset and create a model for the same. This example is also available in the example folder in the project.Create the modelI trained a DecisionTreeClassifier on the iris dataset which requires 4 features — Sepal Length, Sepal Width, Petal Length and Petal Width. Then, I saved the model to classifier.joblib using joblib.dump(). The classifier can now be used to predict on new data.Update the serviceNext, I opened the file app.py in a text editor (Sublime Text is one). I uncommented the line classifier = joblib.load(‘classifier.joblib’) so that the variable classifier now holds the trained model.In the post method, I made the following updates:prediction = classifier.predict(np.array(data).reshape(1, -1)) | types = { 0: "Iris Setosa", 1: "Iris Versicolour ", 2: "Iris Virginica"} | response = jsonify({ | "statusCode": 200, | "status": "Prediction made", | "result": "The type of iris plant is: " + types[prediction[0]] | })Firstly, I used classifier.predict() to get the prediction. Next, I created a map for the classes such that 0 means Iris Setosa, 1 means Iris Versicolour and 2 means Iris Virginica. I finally returned the prediction in the result key.Update: As pointed out by Martins Untals, I forgot to mention that we also need to update the model so that it works correctly and has the updated model in Swagger UI.model = app.model('Prediction params', | {'sepalLength': fields.Float(required = True, | description="Sepal Length", | help="Sepal Length cannot be blank"), | 'sepalWidth': fields.Float(required = True, | description="Sepal Width", | help="Sepal Width cannot be blank"), | 'petalLength': fields.Float(required = True, | description="Petal Length", | help="Petal Length cannot be blank"), | 'petalWidth': fields.Float(required = True, | description="Petal Width", | help="Petal Width cannot be blank")})As can be seen in the gist, I’ve updated the field names, their type to Float, description and help text. The same will now be reflected in Swagger too.Updated model in Swagger UIUpdate the UIThe form is made up of columns inside rows. Thus, as I have 4 features, I added 2 columns in 2 rows. The first row will have dropdowns for Sepal Length and Sepal Width. The second row will have dropdowns for Petal Length and Petal Width.I began by creating a list of options for each of these dropdowns.var sepalLengths = [] | for (var i = 4; i <= 7; i = +(i + 0.1).toFixed(1)) { | sepalLengths.push(<option key = {i} value = {i}>{i}</option>); | } | var sepalWidths = [] | for (var i = 2; i <= 4; i = +(i + 0.1).toFixed(1)) { | sepalWidths.push(<option key = {i} value = {i}>{i}</option>); | } | var petalLengths = [] | for (var i = 1; i <= 6; i = +(i + 0.1).toFixed(1)){ | petalLengths.push(<option key = {i} value = {i}>{i}</option>); | } | var petalWidths = [] | for (var i = 0.1; i <= 3; i = +(i + 0.1).toFixed(1)) { | petalWidths.push(<option key = {i} value = {i}>{i}</option>); | }Next, I defined two rows with two columns each. Each dropdown selection will look like the code below:<Form.Group as={Col}> | <Form.Label>Petal Length</Form.Label> | <Form.Control | as="select" | value={formData.petalLength} | name="petalLength" | onChange={this.handleChange}> | {petalLengths} | </Form.Control> | </Form.Group>For each dropdown, we’ll have to update the text inside <Form.Label></Form.Label>. We’ll name each selection group as well. Let’s say the name be petalLength so we set the value as {formData.petalLength} and name as “petalLength”. The options are added using the names we defined above inside <Form.Control></Form.Control> as we can see {petalLengths} above. Two such groups inside a <Form.Row></Form.Row> will make our UI.The state must also be updated with the same names inside formData with the default values as the smallest value of the respective dropdowns. The constructor will look like below. As you can see, the state has been updated to have formData with new keys.constructor(props) { | super(props); | | this.state = { | isLoading: false, | formData: { | sepalLength: 4, | sepalWidth: 2, | petalLength: 1, | petalWidth: 0 | }, | result: "" | }; | }Add a new background image and titleInside app.css, change the link of the background image to your own. I added an image of flowers from Unsplash. I also updated the title to Iris Plant Classifier and the page title too inside index.html file in the public folder.ResultThe application is now ready to use the model. Restart both services after building the UI using npm run build. The application looks like below:Main PageWith some feature values, on pressing the Predict button, the model classifies it as Iris Setosa.With new feature values, the model predicts the plant to be Iris Versicolour.ConclusionAs you can see, in this article, I discussed a ML React App template that will make creating complete ML applications simple and quick.Try the application for your own use case and share your feedback. I’d love to hear from you.

Karan Bhanot June 22, 2020

Flask Libraries Deployment

Build A Web Data Dashboard In Just Minutes With Python Exponentially increase power & accessibility by converting your data visualizations into a web-based dashboard with Plotly Dash.Build a web data dashboard — in just a few lines of Python codeI don’t know about you, but I occasionally find it a little bit intimidating to have to code something. This is doubly so when I’m building something akin to web development rather than doing some local data analysis and visualisation. I’m a competent Python coder, but I wouldn’t call myself a web developer at all, even after having more than dabbled with Django and Flask.Still, converting your data outputs to a web app leads to a few non-trivial improvements for your project.It is just much easier to build in true, powerful interactivity into a web app. It also means that you can control exactly how the data is presented, as the web app can become the de facto report as well as the access point to your data. Lastly, and most importantly, you can exponentially scale the accessibility to your outputs; making them available anywhere, any time. There is always a web browser at a user’s fingertips.So, I bit the bullet and started to do just this with some of my data projects recently, with surprisingly fast speed and efficiency. I converted one of my outputs from this article to a web app here in just a couple of hours.My NBA analytics web app (link)I thought this was rather cool, and wanted to share how this came together in just a few lines of code.As always, I include everything you need to replicate my steps (data & code), and the article is not really about basketball. So do not worry if you are unfamiliar with it, and let’s get going.Before we get startedDataI include the code and data in my GitLab repo here (dash_simple_nbadirectory). So please feel free to play with it / improve upon it.PackagesI assume you’re familiar with python. Even if you’re relatively new, this tutorial shouldn’t be too tricky, though.You’ll need pandas, plotly and dash. Install each (in your virtual environment) with a simple pip install [PACKAGE_NAME].Previously, on Python…For this tutorial, I am simply going to skip *most* of the steps taken to create the local version of our visualisation. If you’re interested in what is going on, take a look at this article:Create effective data visualizations of proportionsBest ways to see individual contributions to a whole and changes over time, at various dataset sizes — (incl…towardsdatascience.comWe will have a recap session, though, so you can see what is happening between plotting the chart locally with Plotly, and how to port that to a web app with Plotly Dash.Load dataI have pre-processed the data, and saved it as a CSV file. It is a collection of player data for the current NBA season (as of 26/Feb/2020), which shows:What share of their team’s shots they are taking, andHow efficient / effective they are at doing it.For this portion, follow along by opening local_plot.py in my repo.Load the data with:all_teams_df = pd.read_csv(‘srcdata/shot_dist_compiled_data_2019_20.csv’)Inspect the data with all_teams_df.head(), and you should see:Each player’s data has been compiled for each minute of the game (excluding overtime), with stats pl_acc and pl_pps being the only exception, as they have been compiled per quarter of the game (for each 12 minute period).The dataframe contains all NBA players, so let’s break it down to a manageable size, by filtering for a team. For instance, the New Orleans Pelicans’ players can be chosen with:all_teams_df[all_teams_df.group == 'NOP']Then, our data can be visualised in Plotly, as below:import plotly.express as px fig = px.scatter(all_teams_df[all_teams_df.group == 'NOP'], x='min_mid', y='player', size='shots_freq', color='pl_pps') fig.show()Visualised player data for New Orlean PelicansAt the risk of doing this:How to Draw a Horse — Van Oktop (Tweet)I do add a few small details to my chart, to produce this version of the same graph.Same chart, with a few ‘small details’ added (& different team).This is the code that I used to do it.Now, while it’s a lot of formatting code, I thought it useful to show you how I did it, because we are going to be re-using these functions in our Dash version of the code.def clean_chart_format(fig): import plotly.graph_objects as go fig.update_layout( paper_bgcolor="white", plot_bgcolor="white", annotations=[ go.layout.Annotation( x=0.9, y=1.02, showarrow=False, text="Twitter: @_jphwang", xref="paper", yref="paper", textangle=0 ), ], font=dict( family="Arial, Tahoma, Helvetica", size=10, color="#404040" ), margin=dict( t=20 ) ) fig.update_traces(marker=dict(line=dict(width=1, color='Navy')), selector=dict(mode='markers')) fig.update_coloraxes( colorbar=dict( thicknessmode="pixels", thickness=15, outlinewidth=1, outlinecolor='#909090', lenmode="pixels", len=300, yanchor="top", y=1, )) fig.update_yaxes(showgrid=True, gridwidth=1, tickson='boundaries', gridcolor='LightGray', fixedrange=True) fig.update_xaxes(showgrid=True, gridwidth=1, gridcolor='LightGray', fixedrange=True) return True def make_shot_dist_chart(input_df, color_continuous_scale=None, size_col='shots_count', col_col='pl_acc', range_color=None): max_bubble_size = 15 if color_continuous_scale is None: color_continuous_scale = px.colors.diverging.RdYlBu_r if range_color is None: range_color = [min(input_df[col_col]), max(input_df[col_col])] fig = px.scatter( input_df, x='min_mid', y='player', size=size_col, color=col_col, color_continuous_scale=color_continuous_scale, range_color=range_color, range_x=[0, 49], range_y=[-1, len(input_df.player.unique())], hover_name='player', hover_data=['min_start', 'min_end', 'shots_count', 'shots_made', 'shots_freq', 'shots_acc', ], render_mode='svg' ) fig.update_coloraxes(colorbar=dict(title='Points per<BR>100 shots')) fig.update_traces(marker=dict(sizeref=2. * 30 / (max_bubble_size ** 2))) fig.update_yaxes(title="Player") fig.update_xaxes(title='Minute', tickvals=list(range(0, 54, 6))) return fig fig = make_shot_dist_chart( all_teams_df[all_teams_df.group == 'SAS'], col_col='pl_pps', range_color=[90, 120], size_col='shots_freq') clean_chart_format(fig) fig.update_layout(height=500, width=1250) fig.show()Now, let’s get to the main event — how to create a web app out of these plots.Into the World Wide WebYou can read more about Plotly Dash here, but for now all you need to know that it is an open-source software package developed to abstract away the difficulties in putting your visualisations on the web.It works with Flask under the hood, and you can happily reuse most of the code that you used to develop plots in plotly.py.This is the simple version that I put together:import pandas as pd import dash import dash_core_components as dcc import dash_html_components as html from dash.dependencies import Input, Output all_teams_df = pd.read_csv('srcdata/shot_dist_compiled_data_2019_20.csv') app = dash.Dash(__name__) server = app.server team_names = all_teams_df.group.unique() team_names.sort() app.layout = html.Div([ html.Div([dcc.Dropdown(id='group-select', options=[{'label': i, 'value': i} for i in team_names], value='TOR', style={'width': '140px'})]), dcc.Graph('shot-dist-graph', config={'displayModeBar': False})]) @app.callback( Output('shot-dist-graph', 'figure'), [Input('group-select', 'value')] ) def update_graph(grpname): import plotly.express as px return px.scatter(all_teams_df[all_teams_df.group == grpname], x='min_mid', y='player', size='shots_freq', color='pl_pps') if __name__ == '__main__': app.run_server(debug=False)Try it out! It should open this plot on your browser.Our first Dash app!What’s the big deal? Well, for one — it is a live web app, in under 25 lines of code. And notice the drop-down menu on the top left? Try changing the values on it, and watch the graph change *magically*.Go on, I’ll wait.Okay? Done.Let’s briefly go through the code.At a high level, what I’m doing here is to:Initialise a Dash app;Get a list of available team names, and provide it to a dropdown menu (with DOM id group-select) with a default value or ‘TOR’;Instantiate a Graph object as the shot-dist-graph identifier within Dash; andCreate a callback function where if any of the values are changed, it will call the update_graph function and pass the returned object to the Output.If you take a look at the code, so many of what is probably trivial for web devs but annoying for me is abstracted away.dcc.Graph wraps the figure object from plotly.py into my web app and HTML components like divs can be called and set up conveniently with html.Div objects.Most gratifying for me personally is that Input objects and callbacks from those inputs are declaratively set up, and I can avoid having to deal with things like HTML forms or JavaScript.And the resulting app still works beautifully. The graph is updated the moment that the pulldown menu is used to select another value.And we did all that in fewer than 25 lines of code.Why Dash?At this point, you might be asking — why Dash? We can do all this with a JS framework front end, and Flask, or any one of myriad other combinations.To someone like me who prefers the comfort of Python than natively dealing with HTML and CSS, using Dash abstracts away a lot of stuff that doesn’t add a lot of value to the end product.Take, for instance, a version of this app that includes further formatting and notes for the audience:(It is simple_dash_w_format.py in the git repo)def clean_chart_format(fig): fig.update_layout( paper_bgcolor="white", plot_bgcolor="white", annotations=[ go.layout.Annotation( x=0.9, y=1.02, showarrow=False, text="Twitter: @_jphwang", xref="paper", yref="paper", textangle=0 ), ], font=dict( family="Arial, Tahoma, Helvetica", size=10, color="#404040" ), margin=dict( t=20 ) ) fig.update_traces(marker=dict(line=dict(width=1, color='Navy')), selector=dict(mode='markers')) fig.update_coloraxes( colorbar=dict( thicknessmode="pixels", thickness=15, outlinewidth=1, outlinecolor='#909090', lenmode="pixels", len=300, yanchor="top", y=1, )) fig.update_yaxes(showgrid=True, gridwidth=1, tickson='boundaries', gridcolor='LightGray', fixedrange=True) fig.update_xaxes(showgrid=True, gridwidth=1, gridcolor='LightGray', fixedrange=True) return True def make_shot_dist_chart(input_df, color_continuous_scale=None, size_col='shots_count', col_col='pl_acc', range_color=None): max_bubble_size = 15 if color_continuous_scale is None: color_continuous_scale = px.colors.diverging.RdYlBu_r if range_color is None: range_color = [min(input_df[col_col]), max(input_df[col_col])] fig = px.scatter( input_df, x='min_mid', y='player', size=size_col, color=col_col, color_continuous_scale=color_continuous_scale, range_color=range_color, range_x=[0, 49], range_y=[-1, len(input_df.player.unique())], hover_name='player', hover_data=['min_start', 'min_end', 'shots_count', 'shots_made', 'shots_freq', 'shots_acc', ], render_mode='svg' ) fig.update_coloraxes(colorbar=dict(title='Points per<BR>100 shots')) fig.update_traces(marker=dict(sizeref=2. * 30 / (max_bubble_size ** 2))) fig.update_yaxes(title="Player") fig.update_xaxes(title='Minute', tickvals=list(range(0, 54, 6))) return fig app.title = 'Dash Demo - NBA' team_names = all_teams_df.group.unique() team_names.sort() app.layout = html.Div([ html.Div([ dcc.Markdown( """ #### Shot Frequencies & Efficiencies (2019-20 NBA Season) This page compares players based on shot *frequency* and *efficiency*, divided up into minutes of regulation time for each team. Use the pulldown to select a team, or select 'Leaders' to see leaders from each team. *Notes*: * **Frequency**: A team's shots a player is taking, indicated by **size**. * **Efficiency**: Points scored per 100 shots, indicated by **colour** (red == better, blue == worse). * Players with <1% of team shots are shown under 'Others' """ ), html.P([html.Small("See more data / NBA analytics content, find me on "), html.A(html.Small("twitter"), href="https://twitter.com/_jphwang", title="twitter"), html.Small("!")]), ]), html.Div([ dcc.Dropdown( id='group-select', options=[{'label': i, 'value': i} for i in team_names], value='TOR', style={'width': '140px'} ) ]), dcc.Graph( 'shot-dist-graph', config={'displayModeBar': False} ) ]) @app.callback( Output('shot-dist-graph', 'figure'), [Input('group-select', 'value')] ) def update_graph(grpname): fig = make_shot_dist_chart( all_teams_df[all_teams_df.group == grpname], col_col='pl_pps', range_color=[90, 120], size_col='shots_freq') clean_chart_format(fig) if len(grpname) > 3: fig.update_layout(height=850, width=1250) else: fig.update_layout(height=500, width=1250) return figMost of the changes are cosmetic, but I will note that here, I just write the body text in Markdown, and simply carry over my formatting functions from Plotly to be used in the formatting the graphs in Dash.This saves me a tremendous amount of time between doing data analysis and visualisation to deployment to clients’ views.All in all, from starting with my initial graph, I think it probably took less than an hour to deploy it to Heroku. Which is pretty amazing.I will get into more advanced features of Dash, and actually doing some cool things with it functionality-wise, but I was very happy with this outcome in terms of ease and speed.Try it out yourself — I think that you’d be very impressed. Next time, I plan to write about some really cool things you can do with Dash, and building truly interactive dashboards.If you liked this, say 👋 / follow on twitter, or follow for updates. This is the article that the data viz is based on:

Admond Lee May 15, 2020

DataSource.ai Blogs

Newsletter