1. Prerequisites and Environment setup<\/h2>\n
This tutorial is carried out in Anaconda Navigator (Python version \u2013 3.8.3) on Windows Operating System. The following packages need to be installed before you continue with the tutorial \u2013<\/p>\n
- \n
- \n
Pandas<\/p>\n<\/li>\n
- \n
NumPy<\/p>\n<\/li>\n
- \n
tensorflow<\/p>\n<\/li>\n
- \n
nltk<\/p>\n<\/li>\n
- \n
csv<\/p>\n<\/li>\n
- \n
griddb_python<\/p>\n<\/li>\n
- \n
matplotlib<\/p>\n<\/li>\n<\/ol>\n
You can install these packages in Conda\u2019s virtual environment using
conda install package-name<\/code>. In case you are using Python directly via terminal\/command prompt,pip install package-name<\/code> will do the work.<\/p>\nGridDB installation<\/h3>\n
While loading the dataset, this tutorial will cover two methods \u2013 Using GridDB as well as Using Pandas. To access GridDB using Python, the following packages also need to be installed beforehand:<\/p>\n
- \n
- GridDB C-client<\/a><\/li>\n
- SWIG (Simplified Wrapper and Interface Generator)<\/li>\n
- GridDB Python Client<\/a><\/li>\n<\/ol>\n
2. Dataset Overview<\/h2>\n
Text documents are one of the richest sources of data for businesses.<\/p>\n
We\u2019ll use a public dataset from the BBC comprised of 2225 articles, each labeled under one of 5 categories: business, entertainment, politics, sport or tech.<\/p>\n
The dataset used in this project is the BBC News Raw Dataset. It can be downloaded from here (
http:\/\/mlg.ucd.ie\/datasets\/bbc.html<\/code>).<\/p>\n3. Importing Required Libraries<\/h2>\n
\nimport griddb_python as griddb\nimport csv\nimport tensorflow as tf\nimport numpy as np\nimport pandas as pd\nfrom tensorflow.keras.preprocessing.sequence import pad_sequences\nfrom tensorflow.keras.preprocessing.text import Tokenizer\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Dense, Flatten, LSTM, Dropout, Activation, Embedding, Bidirectional\nimport nltk\nfrom nltk.corpus import stopwords\nimport matplotlib.pyplot as plt<\/code><\/pre>\n<\/div>\n4. Loading the Dataset<\/h2>\n
Let\u2019s proceed and load the dataset into our notebook.<\/p>\n
4.a Using GridDB<\/h3>\n
Toshiba GridDB\u2122 is a highly scalable NoSQL database best suited for IoT and Big Data. The foundation of GridDB\u2019s principles is based upon offering a versatile data store that is optimized for IoT, provides high scalability, tuned for high performance, and ensures high reliability.<\/p>\n
To store large amounts of data, a CSV file can be cumbersome. GridDB serves as a perfect alternative as it in open-source and a highly scalable database. GridDB is a scalable, in-memory, No SQL database which makes it easier for you to store large amounts of data. If you are new to GridDB, a tutorial on reading and writing to GridDB<\/a> can be useful.<\/p>\n
Assuming that you have already set up your database, we will now write the SQL query in python to load our dataset.<\/p>\n
\nsql_statement = ('SELECT * FROM bbc-text')\ndataset = pd.read_sql_query(sql_statement, cont)<\/code><\/pre>\n<\/div>\nNote that the
cont<\/code> variable has the container information where our data is stored. Replace thebbc-text<\/code> with the name of your container. More info can be found in this tutorial reading and writing to GridDB<\/a>.<\/p>\nWhen it comes to IoT and Big Data use cases, GridDB clearly stands out among other databases in the Relational and NoSQL space. Overall, GridDB offers multiple reliability features for mission-critical applications that require high availability and data retention.<\/p>\n
4.b Using With statement<\/h3>\n
In Python you need to give access to a file by opening it. You can do it by using the open() function. Open returns a file object, which has methods and attributes for getting information about and manipulating the opened file. Both of the above methods will lead to the same output as the data is loaded in the form of a pandas dataframe using either of the methods.<\/p>\n
We import ntlk library and import the stopwords function. We will set the stopwords for English language. These are the samples for English stopwords: has, hasn\u2019t, and, aren\u2019t, because, each, during.<\/p>\n
The process of converting data to something a computer can understand is referred to as pre-processing. One of the major forms of pre-processing is to filter out useless data. In natural language processing, useless words (data), are referred to as stop words.<\/p>\n
A stop word is a commonly used word (such as \u201cthe\u201d, \u201ca\u201d, \u201can\u201d, \u201cin\u201d) that a search engine has been programmed to ignore, both when indexing entries for searching and when retrieving them as the result of a search query. We would not want these words to take up space in our database, or taking up valuable processing time. For this, we can remove them easily, by storing a list of words that you consider to stop words.<\/p>\n
\nnltk.download('stopwords')\nSTOPWORDS = set(stopwords.words('english'))<\/code><\/pre>\n<\/div>\n\n#We populate the list of articles and labels from the data and also remove the stopwords.\narticles = []\nlabels = []\n\nwith open(\"bbc-text.csv\", 'r') as csvfile:\n reader = csv.reader(csvfile, delimiter=',')\n next(reader)\n for row in reader:\n labels.append(row[0])\n article = row[1]\n for word in STOPWORDS:\n token = ' ' + word + ' '\n article = article.replace(token, ' ')\n article = article.replace(' ', ' ')\n articles.append(article)<\/code><\/pre>\n<\/div>\nWe set the hyper-Parameters that are required to build and train the model.<\/p>\n
\nvocab_size = 5000 \nembedding_dim = 64\nmax_length = 200\ntrunc_type = 'post'\npadding_type = 'post'\noov_tok = '<oov>' # OOV = Out of Vocabulary\ntraining_portion = 0.8<\/oov><\/code><\/pre>\n<\/div>\nOnce the dataset is loaded, let us now explore the dataset. We’ll print the first 10 rows of this dataset using head() function.<\/p>\n
5. Data Cleaning and Preprocessing<\/h2>\n
\narticles[:1]<\/code><\/pre>\n<\/div>\n['tv future hands viewers home theatre systems plasma high-definition tvs digital video recorders moving living room way people watch tv radically different five years time. according expert panel gathered annual consumer electronics show las vegas discuss new technologies impact one favourite pastimes. us leading trend programmes content delivered viewers via home networks cable satellite telecoms companies broadband service providers front rooms portable devices. one talked-about technologies ces digital personal video recorders (dvr pvr). set-top boxes like us tivo uk sky+ system allow people record store play pause forward wind tv programmes want. essentially technology allows much personalised tv. also built-in high-definition tv sets big business japan us slower take europe lack high-definition programming. people forward wind adverts also forget abiding network channel schedules putting together a-la-carte entertainment. us networks cable satellite companies worried means terms advertising revenues well brand identity viewer loyalty channels. although us leads technology moment also concern raised europe particularly growing uptake services like sky+. happens today see nine months years time uk adam hume bbc broadcast futurologist told bbc news website. likes bbc issues lost advertising revenue yet. pressing issue moment commercial uk broadcasters brand loyalty important everyone. talking content brands rather network brands said tim hanlon brand communications firm starcom mediavest. reality broadband connections anybody producer content. added: challenge hard promote programme much choice. means said stacey jolna senior vice president tv guide tv group way people find content want watch simplified tv viewers. means networks us terms channels could take leaf google book search engine future instead scheduler help people find want watch. kind channel model might work younger ipod generation used taking control gadgets play them. might suit everyone panel recognised. older generations comfortable familiar schedules channel brands know getting. perhaps want much choice put hands mr hanlon suggested. end kids diapers pushing buttons already - everything possible available said mr hanlon. ultimately consumer tell market want. 50 000 new gadgets technologies showcased ces many enhancing tv-watching experience. high-definition tv sets everywhere many new models lcd (liquid crystal display) tvs launched dvr capability built instead external boxes. one example launched show humax 26-inch lcd tv 80-hour tivo dvr dvd recorder. one us biggest satellite tv companies directtv even launched branded dvr show 100-hours recording capability instant replay search function. set pause rewind tv 90 hours. microsoft chief bill gates announced pre-show keynote speech partnership tivo called tivotogo means people play recorded programmes windows pcs mobile devices. reflect increasing trend freeing multimedia people watch want want.']\n<\/code><\/pre>\n\nlabels[:1]<\/code><\/pre>\n<\/div>\n\n['tech']<\/code><\/pre>\n<\/div>\nNow, let’s proceed to building and evaluating machine learning models on our credit card dataset. We’ll first create
features<\/code> andlabels<\/code> for our model and split them into train and test samples. Test size has been kept as 20% of the total dataset size.<\/p>\nWe need to split them into training set and validation set. We set 80% (training_portion = .8) for training and another 20% for validation.<\/p>\n
\ntrain_size = int(len(articles) * training_portion)\n\ntrain_articles = articles[0: train_size]\ntrain_labels = labels[0: train_size]\n\nvalidation_articles = articles[train_size:]\nvalidation_labels = labels[train_size:]<\/code><\/pre>\n<\/div>\n5.a Tokenization<\/h3>\n
Tokenization is set with num_words equal to vocab_size (5000) and oov_token equal to ‘
‘. The method fits_on_texts is called on train_articles. By using word frequency, this method creates the vocabulary index. In the example given, “The cat sat on the mat. “, it will create a dictionary {‘ ‘: 1, ‘cat’: 3, ‘mat’: 6, ‘on’: 5, ‘sat’: 4, ‘the’: 2}.<\/p>\n \ntokenizer = Tokenizer(num_words = vocab_size, oov_token=oov_tok)\ntokenizer.fit_on_texts(train_articles)\nword_index = tokenizer.word_index<\/code><\/pre>\n<\/div>\nThe oov_token is the value \u2018
\u2019 that we put if the word is not listed in the dictionary.<\/oov><\/p>\n 5.b Convert to Sequences<\/h3>\n
Tokenization is followed by the method text_to_sequences. It converts each text in texts into an integer sequence. The method basically takes each word in the text and replaces it with its corresponding integer from the dictionary tokenizer.word_index. If the word is not in the dictionary, it will be assigned value of 1.<\/p>\n
\ntrain_sequences = tokenizer.texts_to_sequences(train_articles)<\/code><\/pre>\n<\/div>\n5.c Sequence Truncation and Padding<\/h3>\n
When we train them for NLP, we need to make those sequences the same size (concrete shape). To make sure that all sequences are the same size, we will use padding and truncate them.<\/p>\n
\ntrain_padded = pad_sequences(train_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)<\/code><\/pre>\n<\/div>\nWe will apply tokenization, convert to sequences and padding\/truncating to train_articles and validation_articles.<\/p>\n
\ntokenizer = Tokenizer(num_words = vocab_size, oov_token=oov_tok)\ntokenizer.fit_on_texts(train_articles)\nword_index = tokenizer.word_index\n\ntrain_sequences = tokenizer.texts_to_sequences(train_articles)\ntrain_padded = pad_sequences(train_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n\nvalidation_sequences = tokenizer.texts_to_sequences(validation_articles)\nvalidation_padded = pad_sequences(validation_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)<\/code><\/pre>\n<\/div>\nAs before, we need to do the same thing here as we did with the features and articles. The model does not understand words, so we need to convert the label to numbers. We tokenize and convert to sequence the same as before. When doing the tokenization, we don\u2019t indicate the vocab size and oov_token.<\/p>\n
\nlabel_tokenizer = Tokenizer()\nlabel_tokenizer.fit_on_texts(labels)\n\ntraining_label_seq = np.array(label_tokenizer.texts_to_sequences(train_labels))\nvalidation_label_seq = np.array(label_tokenizer.texts_to_sequences(validation_labels))<\/code><\/pre>\n<\/div>\n6. Machine Learning Model Building<\/h2>\n
Now we are ready to create the Neural Network model. The model architecture consist of the following layers:<\/p>\n
\nmodel = Sequential()\n\nmodel.add(Embedding(vocab_size, embedding_dim))\nmodel.add(Dropout(0.5))\nmodel.add(Bidirectional(LSTM(embedding_dim)))\nmodel.add(Dense(6, activation='softmax'))\n\nmodel.summary()<\/code><\/pre>\n<\/div>\n\nModel: \"sequential\"\n _________________________________________________________________\n Layer (type) Output Shape Param # \n =================================================================\n embedding (Embedding) (None, None, 64) 320000 \n _________________________________________________________________\n dropout (Dropout) (None, None, 64) 0 \n _________________________________________________________________\n bidirectional (Bidirectional (None, 128) 66048 \n _________________________________________________________________\n dense (Dense) (None, 6) 774 \n =================================================================\n Total params: 386,822\n Trainable params: 386,822\n Non-trainable params: 0\n _________________________________________________________________<\/code><\/pre>\n<\/div>\nWe then compile the model to configure the training process with the loss sparse_categorical_crossentropy since we didn\u2019t one-hot encode the labels. We use Adam optimizer.<\/p>\n
\nopt = tf.keras.optimizers.Adam(learning_rate=0.001, decay=1e-6)\nmodel.compile(loss='sparse_categorical_crossentropy', optimizer=opt, metrics=['accuracy'])<\/code><\/pre>\n<\/div>\n\nnum_epochs = 12\nhistory = model.fit(train_padded, training_label_seq, epochs=num_epochs, validation_data=(validation_padded, validation_label_seq), verbose=2)<\/code><\/pre>\n<\/div>\n\nEpoch 1\/12\n 56\/56 - 8s - loss: 1.6055 - accuracy: 0.2949 - val_loss: 1.4597 - val_accuracy: 0.3191\n Epoch 2\/12\n 56\/56 - 5s - loss: 1.0623 - accuracy: 0.5854 - val_loss: 0.7767 - val_accuracy: 0.8000\n Epoch 3\/12\n 56\/56 - 5s - loss: 0.6153 - accuracy: 0.7989 - val_loss: 0.7209 - val_accuracy: 0.7910\n Epoch 4\/12\n 56\/56 - 5s - loss: 0.3402 - accuracy: 0.9101 - val_loss: 0.5048 - val_accuracy: 0.8135\n Epoch 5\/12\n 56\/56 - 6s - loss: 0.1731 - accuracy: 0.9685 - val_loss: 0.1699 - val_accuracy: 0.9618\n Epoch 6\/12\n 56\/56 - 6s - loss: 0.0448 - accuracy: 0.9955 - val_loss: 0.1592 - val_accuracy: 0.9663\n Epoch 7\/12\n 56\/56 - 6s - loss: 0.0333 - accuracy: 0.9966 - val_loss: 0.1428 - val_accuracy: 0.9663\n Epoch 8\/12\n 56\/56 - 5s - loss: 0.0400 - accuracy: 0.9927 - val_loss: 0.1245 - val_accuracy: 0.9685\n Epoch 9\/12\n 56\/56 - 6s - loss: 0.0178 - accuracy: 0.9972 - val_loss: 0.1179 - val_accuracy: 0.9685\n Epoch 10\/12\n 56\/56 - 5s - loss: 0.0135 - accuracy: 0.9972 - val_loss: 0.1557 - val_accuracy: 0.9573\n Epoch 11\/12\n 56\/56 - 5s - loss: 0.0264 - accuracy: 0.9983 - val_loss: 0.1193 - val_accuracy: 0.9685\n Epoch 12\/12\n 56\/56 - 6s - loss: 0.0102 - accuracy: 0.9994 - val_loss: 0.1306 - val_accuracy: 0.9663<\/code><\/pre>\n<\/div>\nWe plot the history for accuracy and loss and see if there is overfitting.<\/p>\n
\ndef plot_graphs(history, string):\n plt.plot(history.history[string])\n plt.plot(history.history['val_'+string])\n plt.xlabel(\"Epoch Count\")\n plt.ylabel(string)\n plt.legend([string, 'val_'+string])\n plt.show()\n \nplot_graphs(history, \"accuracy\")\nplot_graphs(history, \"loss\")<\/code><\/pre>\n<\/div>\n![\"\"](\"https:\/\/griddb.net\/wp-content\/uploads\/2022\/03\/output_49_0.png\")
<\/a><\/p>\n![\"\"](\"https:\/\/griddb.net\/wp-content\/uploads\/2022\/03\/output_49_1.png\")
<\/a><\/p>\nFinally, we call the method predict() to perform prediction on a sample text.<\/p>\n
- GridDB C-client<\/a><\/li>\n