This week

[x] Implement an actual FL algorithm on our data
[ ] Test the Keras and the TFF implementations of the same model

We will work on this notebook:

First, we will work on a normal centralised Keras model using our taxi dataset, then we will try to federate it and check the performance.

As always, we need first to import and install all the dependencies we need:

#@test {"skip": true}
!pip install --quiet --upgrade tensorflow_federated

%load_ext tensorboard

!pip install -q sklearn

I got some problems before with packages, so now I install all the dependencies first and then import them:

# Import tensorflow and libraries

import numpy as np
import pandas as pd
import collections

import tensorflow as tf
tf.compat.v1.enable_v2_behavior()

import tensorflow_federated as tff
np.random.seed(0)
tff.federated_computation(lambda: 'Lets start with FL!')()

from tensorflow import feature_column
from tensorflow.keras import layers
from sklearn.model_selection import train_test_split

import os
import time
import sys

from tqdm import tqdm
from google.colab import drive
drive.mount('/content/gdrive')
baseURL = '/content/gdrive/My Drive/TFF/';
directory = baseURL + 'test'
#tf.gfile.MkDir(directory)

As always we are gonna use G Drive.

Keras Centralized mode

First we need to take care of the dataset:

Dataset

# read the dataset from Drive
df = pd.read_csv(baseURL + "trips_1000000.csv")
# We will use just 100.000 rows to make it feasible
df = df.head(100000)
df.head()

# Check dtypes of the attributes
df.dtypes

# Output
medallion               object
pickup_week_day          int64
pickup_hour              int64
pickup_month             int64
pickup_location_id     float64
dropoff_location_id    float64
dtype: object

As before we can drop the medallion because for a centralized model is not required:

# Drop the medallion, it is not needed for this example
df.drop(['medallion'], axis=1, inplace=True)