Neural Language Model: A Start

• How to Develop Word-Based Neural Language Models in Python with Keras

• English texts

• Three methods to build a neural language model:

  • bigram

  • trigram

  • line-based

Bigram Model

from numpy import array
from keras.preprocessing.text import Tokenizer
from keras.utils import to_categorical, plot_model
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from keras.layers import Embedding

# generate a sequence from the model
def generate_seq(model, tokenizer, seed_text, n_words):
	in_text, result = seed_text, seed_text
	# generate a fixed number of words
	for _ in range(n_words):
		# encode the text as integer
		encoded = tokenizer.texts_to_sequences([in_text])[0]
		encoded = array(encoded)
		# predict a word in the vocabulary
		yhat = model.predict_classes(encoded, verbose=0)
		# map predicted word index to word
		out_word = ''
		for word, index in tokenizer.word_index.items():
			if index == yhat:
				out_word = word
				break
		# append to input
		in_text, result = out_word, result + ' ' + out_word
	return result

# source text
data = """ Jack and Jill went up the hill\n
		To fetch a pail of water\n
		Jack fell down and broke his crown\n
		And Jill came tumbling after\n """
# integer encode text
tokenizer = Tokenizer()
tokenizer.fit_on_texts([data])
encoded = tokenizer.texts_to_sequences([data])[0]
# determine the vocabulary size
vocab_size = len(tokenizer.word_index) + 1
print('Vocabulary Size: %d' % vocab_size)
# create word -> word sequences
sequences = list()
for i in range(1, len(encoded)):
	sequence = encoded[i-1:i+1]
	sequences.append(sequence)
print('Total Sequences: %d' % len(sequences))
# split into X and y elements
sequences = array(sequences)
X, y = sequences[:,0],sequences[:,1]
# one hot encode outputs
y = to_categorical(y, num_classes=vocab_size)

Vocabulary Size: 22
Total Sequences: 24

# define model
model = Sequential()
model.add(Embedding(vocab_size, 10, input_length=1))
model.add(LSTM(50))
model.add(Dense(vocab_size, activation='softmax'))
print(model.summary())
# compile network
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit network
model.fit(X, y, epochs=500, verbose=2)

Show code cell output Hide code cell output

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding (Embedding)        (None, 1, 10)             220       
_________________________________________________________________
lstm (LSTM)                  (None, 50)                12200     
_________________________________________________________________
dense (Dense)                (None, 22)                1122      
=================================================================
Total params: 13,542
Trainable params: 13,542
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/500
1/1 - 0s - loss: 3.0917 - accuracy: 0.1250
Epoch 2/500
1/1 - 0s - loss: 3.0909 - accuracy: 0.0833
Epoch 3/500
1/1 - 0s - loss: 3.0901 - accuracy: 0.0833
Epoch 4/500
1/1 - 0s - loss: 3.0894 - accuracy: 0.0833
Epoch 5/500
1/1 - 0s - loss: 3.0886 - accuracy: 0.0833
Epoch 6/500
1/1 - 0s - loss: 3.0878 - accuracy: 0.1250
Epoch 7/500
1/1 - 0s - loss: 3.0870 - accuracy: 0.1250
Epoch 8/500
1/1 - 0s - loss: 3.0862 - accuracy: 0.1250
Epoch 9/500
1/1 - 0s - loss: 3.0854 - accuracy: 0.1250
Epoch 10/500
1/1 - 0s - loss: 3.0846 - accuracy: 0.1250
Epoch 11/500
1/1 - 0s - loss: 3.0838 - accuracy: 0.1250
Epoch 12/500
1/1 - 0s - loss: 3.0830 - accuracy: 0.1250
Epoch 13/500
1/1 - 0s - loss: 3.0821 - accuracy: 0.1250
Epoch 14/500
1/1 - 0s - loss: 3.0813 - accuracy: 0.1250
Epoch 15/500
1/1 - 0s - loss: 3.0804 - accuracy: 0.1250
Epoch 16/500
1/1 - 0s - loss: 3.0795 - accuracy: 0.1250
Epoch 17/500
1/1 - 0s - loss: 3.0786 - accuracy: 0.1250
Epoch 18/500
1/1 - 0s - loss: 3.0777 - accuracy: 0.1250
Epoch 19/500
1/1 - 0s - loss: 3.0768 - accuracy: 0.1250
Epoch 20/500
1/1 - 0s - loss: 3.0759 - accuracy: 0.1250
Epoch 21/500
1/1 - 0s - loss: 3.0749 - accuracy: 0.1250
Epoch 22/500
1/1 - 0s - loss: 3.0739 - accuracy: 0.1250
Epoch 23/500
1/1 - 0s - loss: 3.0729 - accuracy: 0.1250
Epoch 24/500
1/1 - 0s - loss: 3.0718 - accuracy: 0.1250
Epoch 25/500
1/1 - 0s - loss: 3.0708 - accuracy: 0.1250
Epoch 26/500
1/1 - 0s - loss: 3.0697 - accuracy: 0.1250
Epoch 27/500
1/1 - 0s - loss: 3.0685 - accuracy: 0.1250
Epoch 28/500
1/1 - 0s - loss: 3.0674 - accuracy: 0.1250
Epoch 29/500
1/1 - 0s - loss: 3.0662 - accuracy: 0.1250
Epoch 30/500
1/1 - 0s - loss: 3.0650 - accuracy: 0.1250
Epoch 31/500
1/1 - 0s - loss: 3.0637 - accuracy: 0.1250
Epoch 32/500
1/1 - 0s - loss: 3.0624 - accuracy: 0.1250
Epoch 33/500
1/1 - 0s - loss: 3.0611 - accuracy: 0.1250
Epoch 34/500
1/1 - 0s - loss: 3.0597 - accuracy: 0.1250
Epoch 35/500
1/1 - 0s - loss: 3.0583 - accuracy: 0.1250
Epoch 36/500
1/1 - 0s - loss: 3.0568 - accuracy: 0.1250
Epoch 37/500
1/1 - 0s - loss: 3.0553 - accuracy: 0.1250
Epoch 38/500
1/1 - 0s - loss: 3.0538 - accuracy: 0.1250
Epoch 39/500
1/1 - 0s - loss: 3.0522 - accuracy: 0.1250
Epoch 40/500
1/1 - 0s - loss: 3.0505 - accuracy: 0.1250
Epoch 41/500
1/1 - 0s - loss: 3.0489 - accuracy: 0.1250
Epoch 42/500
1/1 - 0s - loss: 3.0471 - accuracy: 0.1250
Epoch 43/500
1/1 - 0s - loss: 3.0453 - accuracy: 0.1250
Epoch 44/500
1/1 - 0s - loss: 3.0435 - accuracy: 0.1250
Epoch 45/500
1/1 - 0s - loss: 3.0416 - accuracy: 0.1250
Epoch 46/500
1/1 - 0s - loss: 3.0396 - accuracy: 0.1250
Epoch 47/500
1/1 - 0s - loss: 3.0376 - accuracy: 0.1250
Epoch 48/500
1/1 - 0s - loss: 3.0355 - accuracy: 0.1250
Epoch 49/500
1/1 - 0s - loss: 3.0333 - accuracy: 0.1250
Epoch 50/500
1/1 - 0s - loss: 3.0311 - accuracy: 0.1250
Epoch 51/500
1/1 - 0s - loss: 3.0288 - accuracy: 0.1250
Epoch 52/500
1/1 - 0s - loss: 3.0264 - accuracy: 0.1250
Epoch 53/500
1/1 - 0s - loss: 3.0240 - accuracy: 0.1250
Epoch 54/500
1/1 - 0s - loss: 3.0215 - accuracy: 0.1250
Epoch 55/500
1/1 - 0s - loss: 3.0189 - accuracy: 0.1250
Epoch 56/500
1/1 - 0s - loss: 3.0162 - accuracy: 0.1250
Epoch 57/500
1/1 - 0s - loss: 3.0134 - accuracy: 0.1250
Epoch 58/500
1/1 - 0s - loss: 3.0106 - accuracy: 0.1250
Epoch 59/500
1/1 - 0s - loss: 3.0077 - accuracy: 0.1250
Epoch 60/500
1/1 - 0s - loss: 3.0047 - accuracy: 0.1250
Epoch 61/500
1/1 - 0s - loss: 3.0015 - accuracy: 0.1250
Epoch 62/500
1/1 - 0s - loss: 2.9983 - accuracy: 0.1250
Epoch 63/500
1/1 - 0s - loss: 2.9950 - accuracy: 0.1250
Epoch 64/500
1/1 - 0s - loss: 2.9917 - accuracy: 0.1250
Epoch 65/500
1/1 - 0s - loss: 2.9882 - accuracy: 0.1250
Epoch 66/500
1/1 - 0s - loss: 2.9846 - accuracy: 0.1250
Epoch 67/500
1/1 - 0s - loss: 2.9809 - accuracy: 0.1250
Epoch 68/500
1/1 - 0s - loss: 2.9770 - accuracy: 0.1250
Epoch 69/500
1/1 - 0s - loss: 2.9731 - accuracy: 0.1250
Epoch 70/500
1/1 - 0s - loss: 2.9691 - accuracy: 0.1250
Epoch 71/500
1/1 - 0s - loss: 2.9649 - accuracy: 0.1250
Epoch 72/500
1/1 - 0s - loss: 2.9606 - accuracy: 0.1250
Epoch 73/500
1/1 - 0s - loss: 2.9562 - accuracy: 0.1250
Epoch 74/500
1/1 - 0s - loss: 2.9517 - accuracy: 0.1250
Epoch 75/500
1/1 - 0s - loss: 2.9471 - accuracy: 0.1250
Epoch 76/500
1/1 - 0s - loss: 2.9423 - accuracy: 0.1250
Epoch 77/500
1/1 - 0s - loss: 2.9374 - accuracy: 0.1250
Epoch 78/500
1/1 - 0s - loss: 2.9323 - accuracy: 0.1250
Epoch 79/500
1/1 - 0s - loss: 2.9272 - accuracy: 0.1250
Epoch 80/500
1/1 - 0s - loss: 2.9218 - accuracy: 0.1250
Epoch 81/500
1/1 - 0s - loss: 2.9164 - accuracy: 0.1250
Epoch 82/500
1/1 - 0s - loss: 2.9108 - accuracy: 0.1250
Epoch 83/500
1/1 - 0s - loss: 2.9050 - accuracy: 0.1250
Epoch 84/500
1/1 - 0s - loss: 2.8991 - accuracy: 0.1250
Epoch 85/500
1/1 - 0s - loss: 2.8930 - accuracy: 0.1250
Epoch 86/500
1/1 - 0s - loss: 2.8868 - accuracy: 0.1250
Epoch 87/500
1/1 - 0s - loss: 2.8805 - accuracy: 0.1250
Epoch 88/500
1/1 - 0s - loss: 2.8739 - accuracy: 0.1250
Epoch 89/500
1/1 - 0s - loss: 2.8672 - accuracy: 0.1250
Epoch 90/500
1/1 - 0s - loss: 2.8604 - accuracy: 0.1250
Epoch 91/500
1/1 - 0s - loss: 2.8534 - accuracy: 0.1250
Epoch 92/500
1/1 - 0s - loss: 2.8462 - accuracy: 0.1250
Epoch 93/500
1/1 - 0s - loss: 2.8388 - accuracy: 0.1250
Epoch 94/500
1/1 - 0s - loss: 2.8313 - accuracy: 0.1250
Epoch 95/500
1/1 - 0s - loss: 2.8236 - accuracy: 0.1250
Epoch 96/500
1/1 - 0s - loss: 2.8157 - accuracy: 0.1250
Epoch 97/500
1/1 - 0s - loss: 2.8077 - accuracy: 0.1250
Epoch 98/500
1/1 - 0s - loss: 2.7995 - accuracy: 0.1250
Epoch 99/500
1/1 - 0s - loss: 2.7911 - accuracy: 0.1250
Epoch 100/500
1/1 - 0s - loss: 2.7825 - accuracy: 0.1250
Epoch 101/500
1/1 - 0s - loss: 2.7738 - accuracy: 0.1250
Epoch 102/500
1/1 - 0s - loss: 2.7648 - accuracy: 0.1250
Epoch 103/500
1/1 - 0s - loss: 2.7557 - accuracy: 0.1250
Epoch 104/500
1/1 - 0s - loss: 2.7464 - accuracy: 0.1250
Epoch 105/500
1/1 - 0s - loss: 2.7370 - accuracy: 0.1250
Epoch 106/500
1/1 - 0s - loss: 2.7273 - accuracy: 0.1250
Epoch 107/500
1/1 - 0s - loss: 2.7175 - accuracy: 0.1250
Epoch 108/500
1/1 - 0s - loss: 2.7075 - accuracy: 0.1250
Epoch 109/500
1/1 - 0s - loss: 2.6973 - accuracy: 0.1250
Epoch 110/500
1/1 - 0s - loss: 2.6870 - accuracy: 0.1250
Epoch 111/500
1/1 - 0s - loss: 2.6765 - accuracy: 0.1667
Epoch 112/500
1/1 - 0s - loss: 2.6658 - accuracy: 0.1667
Epoch 113/500
1/1 - 0s - loss: 2.6549 - accuracy: 0.1667
Epoch 114/500
1/1 - 0s - loss: 2.6439 - accuracy: 0.1667
Epoch 115/500
1/1 - 0s - loss: 2.6327 - accuracy: 0.1667
Epoch 116/500
1/1 - 0s - loss: 2.6214 - accuracy: 0.1667
Epoch 117/500
1/1 - 0s - loss: 2.6098 - accuracy: 0.1667
Epoch 118/500
1/1 - 0s - loss: 2.5982 - accuracy: 0.1667
Epoch 119/500
1/1 - 0s - loss: 2.5863 - accuracy: 0.1667
Epoch 120/500
1/1 - 0s - loss: 2.5744 - accuracy: 0.1667
Epoch 121/500
1/1 - 0s - loss: 2.5623 - accuracy: 0.1667
Epoch 122/500
1/1 - 0s - loss: 2.5500 - accuracy: 0.1667
Epoch 123/500
1/1 - 0s - loss: 2.5376 - accuracy: 0.1667
Epoch 124/500
1/1 - 0s - loss: 2.5251 - accuracy: 0.1667
Epoch 125/500
1/1 - 0s - loss: 2.5124 - accuracy: 0.2083
Epoch 126/500
1/1 - 0s - loss: 2.4996 - accuracy: 0.2083
Epoch 127/500
1/1 - 0s - loss: 2.4867 - accuracy: 0.2500
Epoch 128/500
1/1 - 0s - loss: 2.4737 - accuracy: 0.2500
Epoch 129/500
1/1 - 0s - loss: 2.4606 - accuracy: 0.2500
Epoch 130/500
1/1 - 0s - loss: 2.4473 - accuracy: 0.2500
Epoch 131/500
1/1 - 0s - loss: 2.4340 - accuracy: 0.2500
Epoch 132/500
1/1 - 0s - loss: 2.4205 - accuracy: 0.2500
Epoch 133/500
1/1 - 0s - loss: 2.4070 - accuracy: 0.2500

Epoch 134/500
1/1 - 0s - loss: 2.3934 - accuracy: 0.2917
Epoch 135/500
1/1 - 0s - loss: 2.3797 - accuracy: 0.3750
Epoch 136/500
1/1 - 0s - loss: 2.3659 - accuracy: 0.3750
Epoch 137/500
1/1 - 0s - loss: 2.3520 - accuracy: 0.3750
Epoch 138/500
1/1 - 0s - loss: 2.3381 - accuracy: 0.3750
Epoch 139/500
1/1 - 0s - loss: 2.3241 - accuracy: 0.3750
Epoch 140/500
1/1 - 0s - loss: 2.3100 - accuracy: 0.3750
Epoch 141/500
1/1 - 0s - loss: 2.2959 - accuracy: 0.4167
Epoch 142/500
1/1 - 0s - loss: 2.2818 - accuracy: 0.4167
Epoch 143/500
1/1 - 0s - loss: 2.2676 - accuracy: 0.4167
Epoch 144/500
1/1 - 0s - loss: 2.2533 - accuracy: 0.4167
Epoch 145/500
1/1 - 0s - loss: 2.2390 - accuracy: 0.4167
Epoch 146/500
1/1 - 0s - loss: 2.2247 - accuracy: 0.4167
Epoch 147/500
1/1 - 0s - loss: 2.2103 - accuracy: 0.4167
Epoch 148/500
1/1 - 0s - loss: 2.1959 - accuracy: 0.4583
Epoch 149/500
1/1 - 0s - loss: 2.1815 - accuracy: 0.5000
Epoch 150/500
1/1 - 0s - loss: 2.1671 - accuracy: 0.5000
Epoch 151/500
1/1 - 0s - loss: 2.1526 - accuracy: 0.5000
Epoch 152/500
1/1 - 0s - loss: 2.1381 - accuracy: 0.5000
Epoch 153/500
1/1 - 0s - loss: 2.1237 - accuracy: 0.5417
Epoch 154/500
1/1 - 0s - loss: 2.1092 - accuracy: 0.5417
Epoch 155/500
1/1 - 0s - loss: 2.0947 - accuracy: 0.5417
Epoch 156/500
1/1 - 0s - loss: 2.0802 - accuracy: 0.5417
Epoch 157/500
1/1 - 0s - loss: 2.0657 - accuracy: 0.6250
Epoch 158/500
1/1 - 0s - loss: 2.0512 - accuracy: 0.6250
Epoch 159/500
1/1 - 0s - loss: 2.0367 - accuracy: 0.6250
Epoch 160/500
1/1 - 0s - loss: 2.0222 - accuracy: 0.6667
Epoch 161/500
1/1 - 0s - loss: 2.0077 - accuracy: 0.6667
Epoch 162/500
1/1 - 0s - loss: 1.9932 - accuracy: 0.6667
Epoch 163/500
1/1 - 0s - loss: 1.9788 - accuracy: 0.6667
Epoch 164/500
1/1 - 0s - loss: 1.9643 - accuracy: 0.6667
Epoch 165/500
1/1 - 0s - loss: 1.9499 - accuracy: 0.6667
Epoch 166/500
1/1 - 0s - loss: 1.9355 - accuracy: 0.6667
Epoch 167/500
1/1 - 0s - loss: 1.9212 - accuracy: 0.6667
Epoch 168/500
1/1 - 0s - loss: 1.9068 - accuracy: 0.6667
Epoch 169/500
1/1 - 0s - loss: 1.8925 - accuracy: 0.6667
Epoch 170/500
1/1 - 0s - loss: 1.8783 - accuracy: 0.7083
Epoch 171/500
1/1 - 0s - loss: 1.8640 - accuracy: 0.7083
Epoch 172/500
1/1 - 0s - loss: 1.8498 - accuracy: 0.7083
Epoch 173/500
1/1 - 0s - loss: 1.8357 - accuracy: 0.7083
Epoch 174/500
1/1 - 0s - loss: 1.8216 - accuracy: 0.7083
Epoch 175/500
1/1 - 0s - loss: 1.8075 - accuracy: 0.7083
Epoch 176/500
1/1 - 0s - loss: 1.7935 - accuracy: 0.7083
Epoch 177/500
1/1 - 0s - loss: 1.7795 - accuracy: 0.7083
Epoch 178/500
1/1 - 0s - loss: 1.7656 - accuracy: 0.7083
Epoch 179/500
1/1 - 0s - loss: 1.7517 - accuracy: 0.7500
Epoch 180/500
1/1 - 0s - loss: 1.7379 - accuracy: 0.7500
Epoch 181/500
1/1 - 0s - loss: 1.7241 - accuracy: 0.7500
Epoch 182/500
1/1 - 0s - loss: 1.7104 - accuracy: 0.7500
Epoch 183/500
1/1 - 0s - loss: 1.6968 - accuracy: 0.7500
Epoch 184/500
1/1 - 0s - loss: 1.6832 - accuracy: 0.7500
Epoch 185/500
1/1 - 0s - loss: 1.6697 - accuracy: 0.7500
Epoch 186/500
1/1 - 0s - loss: 1.6562 - accuracy: 0.7500
Epoch 187/500
1/1 - 0s - loss: 1.6428 - accuracy: 0.7500
Epoch 188/500
1/1 - 0s - loss: 1.6295 - accuracy: 0.7500
Epoch 189/500
1/1 - 0s - loss: 1.6163 - accuracy: 0.7500
Epoch 190/500
1/1 - 0s - loss: 1.6031 - accuracy: 0.7500
Epoch 191/500
1/1 - 0s - loss: 1.5900 - accuracy: 0.7500
Epoch 192/500
1/1 - 0s - loss: 1.5770 - accuracy: 0.7500
Epoch 193/500
1/1 - 0s - loss: 1.5640 - accuracy: 0.7500
Epoch 194/500
1/1 - 0s - loss: 1.5512 - accuracy: 0.7500
Epoch 195/500
1/1 - 0s - loss: 1.5383 - accuracy: 0.7500
Epoch 196/500
1/1 - 0s - loss: 1.5256 - accuracy: 0.7500
Epoch 197/500
1/1 - 0s - loss: 1.5130 - accuracy: 0.7500
Epoch 198/500
1/1 - 0s - loss: 1.5004 - accuracy: 0.7500
Epoch 199/500
1/1 - 0s - loss: 1.4879 - accuracy: 0.7500
Epoch 200/500
1/1 - 0s - loss: 1.4755 - accuracy: 0.7500
Epoch 201/500
1/1 - 0s - loss: 1.4632 - accuracy: 0.7500
Epoch 202/500
1/1 - 0s - loss: 1.4509 - accuracy: 0.7500
Epoch 203/500
1/1 - 0s - loss: 1.4387 - accuracy: 0.7500
Epoch 204/500
1/1 - 0s - loss: 1.4266 - accuracy: 0.7500
Epoch 205/500
1/1 - 0s - loss: 1.4146 - accuracy: 0.7500
Epoch 206/500
1/1 - 0s - loss: 1.4027 - accuracy: 0.7500
Epoch 207/500
1/1 - 0s - loss: 1.3908 - accuracy: 0.7500
Epoch 208/500
1/1 - 0s - loss: 1.3791 - accuracy: 0.7500
Epoch 209/500
1/1 - 0s - loss: 1.3674 - accuracy: 0.7917
Epoch 210/500
1/1 - 0s - loss: 1.3557 - accuracy: 0.7917
Epoch 211/500
1/1 - 0s - loss: 1.3442 - accuracy: 0.7917
Epoch 212/500
1/1 - 0s - loss: 1.3327 - accuracy: 0.7917
Epoch 213/500
1/1 - 0s - loss: 1.3213 - accuracy: 0.7917
Epoch 214/500
1/1 - 0s - loss: 1.3100 - accuracy: 0.7917
Epoch 215/500
1/1 - 0s - loss: 1.2988 - accuracy: 0.7917
Epoch 216/500
1/1 - 0s - loss: 1.2876 - accuracy: 0.7917
Epoch 217/500
1/1 - 0s - loss: 1.2766 - accuracy: 0.7917
Epoch 218/500
1/1 - 0s - loss: 1.2656 - accuracy: 0.7917
Epoch 219/500
1/1 - 0s - loss: 1.2546 - accuracy: 0.8333
Epoch 220/500
1/1 - 0s - loss: 1.2438 - accuracy: 0.8333
Epoch 221/500
1/1 - 0s - loss: 1.2330 - accuracy: 0.8333
Epoch 222/500
1/1 - 0s - loss: 1.2223 - accuracy: 0.8333
Epoch 223/500
1/1 - 0s - loss: 1.2116 - accuracy: 0.8333
Epoch 224/500
1/1 - 0s - loss: 1.2011 - accuracy: 0.8333
Epoch 225/500
1/1 - 0s - loss: 1.1906 - accuracy: 0.8333
Epoch 226/500
1/1 - 0s - loss: 1.1801 - accuracy: 0.8333
Epoch 227/500
1/1 - 0s - loss: 1.1698 - accuracy: 0.8333
Epoch 228/500
1/1 - 0s - loss: 1.1595 - accuracy: 0.8333
Epoch 229/500
1/1 - 0s - loss: 1.1493 - accuracy: 0.8333
Epoch 230/500
1/1 - 0s - loss: 1.1391 - accuracy: 0.8333
Epoch 231/500
1/1 - 0s - loss: 1.1290 - accuracy: 0.8333
Epoch 232/500
1/1 - 0s - loss: 1.1190 - accuracy: 0.8333
Epoch 233/500
1/1 - 0s - loss: 1.1091 - accuracy: 0.8333
Epoch 234/500
1/1 - 0s - loss: 1.0992 - accuracy: 0.8750
Epoch 235/500
1/1 - 0s - loss: 1.0894 - accuracy: 0.8750
Epoch 236/500
1/1 - 0s - loss: 1.0796 - accuracy: 0.8750
Epoch 237/500
1/1 - 0s - loss: 1.0700 - accuracy: 0.8750
Epoch 238/500
1/1 - 0s - loss: 1.0603 - accuracy: 0.8750
Epoch 239/500
1/1 - 0s - loss: 1.0508 - accuracy: 0.8750
Epoch 240/500
1/1 - 0s - loss: 1.0413 - accuracy: 0.8750
Epoch 241/500
1/1 - 0s - loss: 1.0319 - accuracy: 0.8750
Epoch 242/500
1/1 - 0s - loss: 1.0225 - accuracy: 0.8750
Epoch 243/500
1/1 - 0s - loss: 1.0132 - accuracy: 0.8750
Epoch 244/500
1/1 - 0s - loss: 1.0040 - accuracy: 0.8750
Epoch 245/500
1/1 - 0s - loss: 0.9948 - accuracy: 0.8750
Epoch 246/500
1/1 - 0s - loss: 0.9857 - accuracy: 0.8750
Epoch 247/500
1/1 - 0s - loss: 0.9766 - accuracy: 0.8750
Epoch 248/500
1/1 - 0s - loss: 0.9676 - accuracy: 0.8750
Epoch 249/500
1/1 - 0s - loss: 0.9587 - accuracy: 0.8750
Epoch 250/500
1/1 - 0s - loss: 0.9499 - accuracy: 0.8750
Epoch 251/500
1/1 - 0s - loss: 0.9411 - accuracy: 0.8750
Epoch 252/500
1/1 - 0s - loss: 0.9323 - accuracy: 0.8750
Epoch 253/500
1/1 - 0s - loss: 0.9237 - accuracy: 0.8750
Epoch 254/500
1/1 - 0s - loss: 0.9150 - accuracy: 0.8750
Epoch 255/500
1/1 - 0s - loss: 0.9065 - accuracy: 0.8750
Epoch 256/500
1/1 - 0s - loss: 0.8980 - accuracy: 0.8750
Epoch 257/500
1/1 - 0s - loss: 0.8896 - accuracy: 0.8750
Epoch 258/500
1/1 - 0s - loss: 0.8812 - accuracy: 0.8750
Epoch 259/500
1/1 - 0s - loss: 0.8730 - accuracy: 0.8750
Epoch 260/500
1/1 - 0s - loss: 0.8647 - accuracy: 0.8750
Epoch 261/500
1/1 - 0s - loss: 0.8566 - accuracy: 0.8750
Epoch 262/500
1/1 - 0s - loss: 0.8485 - accuracy: 0.8750
Epoch 263/500
1/1 - 0s - loss: 0.8404 - accuracy: 0.8750
Epoch 264/500
1/1 - 0s - loss: 0.8325 - accuracy: 0.8750
Epoch 265/500
1/1 - 0s - loss: 0.8246 - accuracy: 0.8750
Epoch 266/500
1/1 - 0s - loss: 0.8167 - accuracy: 0.8750
Epoch 267/500
1/1 - 0s - loss: 0.8089 - accuracy: 0.8750
Epoch 268/500
1/1 - 0s - loss: 0.8012 - accuracy: 0.8750
Epoch 269/500
1/1 - 0s - loss: 0.7936 - accuracy: 0.8750
Epoch 270/500
1/1 - 0s - loss: 0.7860 - accuracy: 0.8750
Epoch 271/500
1/1 - 0s - loss: 0.7785 - accuracy: 0.8750
Epoch 272/500
1/1 - 0s - loss: 0.7711 - accuracy: 0.8750
Epoch 273/500
1/1 - 0s - loss: 0.7637 - accuracy: 0.8750
Epoch 274/500
1/1 - 0s - loss: 0.7564 - accuracy: 0.8750
Epoch 275/500
1/1 - 0s - loss: 0.7492 - accuracy: 0.8750
Epoch 276/500
1/1 - 0s - loss: 0.7420 - accuracy: 0.8750
Epoch 277/500
1/1 - 0s - loss: 0.7349 - accuracy: 0.8750

Epoch 278/500
1/1 - 0s - loss: 0.7279 - accuracy: 0.8750
Epoch 279/500
1/1 - 0s - loss: 0.7209 - accuracy: 0.8750
Epoch 280/500
1/1 - 0s - loss: 0.7140 - accuracy: 0.8750
Epoch 281/500
1/1 - 0s - loss: 0.7072 - accuracy: 0.8750
Epoch 282/500
1/1 - 0s - loss: 0.7004 - accuracy: 0.8750
Epoch 283/500
1/1 - 0s - loss: 0.6937 - accuracy: 0.8750
Epoch 284/500
1/1 - 0s - loss: 0.6871 - accuracy: 0.8750
Epoch 285/500
1/1 - 0s - loss: 0.6806 - accuracy: 0.8750
Epoch 286/500
1/1 - 0s - loss: 0.6741 - accuracy: 0.8750
Epoch 287/500
1/1 - 0s - loss: 0.6677 - accuracy: 0.8750
Epoch 288/500
1/1 - 0s - loss: 0.6614 - accuracy: 0.8750
Epoch 289/500
1/1 - 0s - loss: 0.6551 - accuracy: 0.8750
Epoch 290/500
1/1 - 0s - loss: 0.6489 - accuracy: 0.8750
Epoch 291/500
1/1 - 0s - loss: 0.6428 - accuracy: 0.8750
Epoch 292/500
1/1 - 0s - loss: 0.6368 - accuracy: 0.8750
Epoch 293/500
1/1 - 0s - loss: 0.6308 - accuracy: 0.8750
Epoch 294/500
1/1 - 0s - loss: 0.6249 - accuracy: 0.8750
Epoch 295/500
1/1 - 0s - loss: 0.6190 - accuracy: 0.8750
Epoch 296/500
1/1 - 0s - loss: 0.6133 - accuracy: 0.8750
Epoch 297/500
1/1 - 0s - loss: 0.6076 - accuracy: 0.8750
Epoch 298/500
1/1 - 0s - loss: 0.6019 - accuracy: 0.8750
Epoch 299/500
1/1 - 0s - loss: 0.5964 - accuracy: 0.8750
Epoch 300/500
1/1 - 0s - loss: 0.5909 - accuracy: 0.8750
Epoch 301/500
1/1 - 0s - loss: 0.5855 - accuracy: 0.8750
Epoch 302/500
1/1 - 0s - loss: 0.5801 - accuracy: 0.8750
Epoch 303/500
1/1 - 0s - loss: 0.5748 - accuracy: 0.8750
Epoch 304/500
1/1 - 0s - loss: 0.5696 - accuracy: 0.8750
Epoch 305/500
1/1 - 0s - loss: 0.5645 - accuracy: 0.8750
Epoch 306/500
1/1 - 0s - loss: 0.5594 - accuracy: 0.8750
Epoch 307/500
1/1 - 0s - loss: 0.5544 - accuracy: 0.8750
Epoch 308/500
1/1 - 0s - loss: 0.5495 - accuracy: 0.8750
Epoch 309/500
1/1 - 0s - loss: 0.5446 - accuracy: 0.8750
Epoch 310/500
1/1 - 0s - loss: 0.5398 - accuracy: 0.8750
Epoch 311/500
1/1 - 0s - loss: 0.5351 - accuracy: 0.8750
Epoch 312/500
1/1 - 0s - loss: 0.5304 - accuracy: 0.8750
Epoch 313/500
1/1 - 0s - loss: 0.5258 - accuracy: 0.8750
Epoch 314/500
1/1 - 0s - loss: 0.5213 - accuracy: 0.8750
Epoch 315/500
1/1 - 0s - loss: 0.5168 - accuracy: 0.8750
Epoch 316/500
1/1 - 0s - loss: 0.5124 - accuracy: 0.8750
Epoch 317/500
1/1 - 0s - loss: 0.5080 - accuracy: 0.8750
Epoch 318/500
1/1 - 0s - loss: 0.5037 - accuracy: 0.8750
Epoch 319/500
1/1 - 0s - loss: 0.4995 - accuracy: 0.8750
Epoch 320/500
1/1 - 0s - loss: 0.4954 - accuracy: 0.8750
Epoch 321/500
1/1 - 0s - loss: 0.4913 - accuracy: 0.8750
Epoch 322/500
1/1 - 0s - loss: 0.4872 - accuracy: 0.8750
Epoch 323/500
1/1 - 0s - loss: 0.4832 - accuracy: 0.8750
Epoch 324/500
1/1 - 0s - loss: 0.4793 - accuracy: 0.8750
Epoch 325/500
1/1 - 0s - loss: 0.4755 - accuracy: 0.8750
Epoch 326/500
1/1 - 0s - loss: 0.4717 - accuracy: 0.8750
Epoch 327/500
1/1 - 0s - loss: 0.4679 - accuracy: 0.8750
Epoch 328/500
1/1 - 0s - loss: 0.4642 - accuracy: 0.8750
Epoch 329/500
1/1 - 0s - loss: 0.4606 - accuracy: 0.8750
Epoch 330/500
1/1 - 0s - loss: 0.4570 - accuracy: 0.8750
Epoch 331/500
1/1 - 0s - loss: 0.4535 - accuracy: 0.8750
Epoch 332/500
1/1 - 0s - loss: 0.4500 - accuracy: 0.8750
Epoch 333/500
1/1 - 0s - loss: 0.4466 - accuracy: 0.8750
Epoch 334/500
1/1 - 0s - loss: 0.4433 - accuracy: 0.8750
Epoch 335/500
1/1 - 0s - loss: 0.4400 - accuracy: 0.8750
Epoch 336/500
1/1 - 0s - loss: 0.4367 - accuracy: 0.8750
Epoch 337/500
1/1 - 0s - loss: 0.4335 - accuracy: 0.8750
Epoch 338/500
1/1 - 0s - loss: 0.4304 - accuracy: 0.8750
Epoch 339/500
1/1 - 0s - loss: 0.4273 - accuracy: 0.8750
Epoch 340/500
1/1 - 0s - loss: 0.4242 - accuracy: 0.8750
Epoch 341/500
1/1 - 0s - loss: 0.4212 - accuracy: 0.8750
Epoch 342/500
1/1 - 0s - loss: 0.4182 - accuracy: 0.8750
Epoch 343/500
1/1 - 0s - loss: 0.4153 - accuracy: 0.8750
Epoch 344/500
1/1 - 0s - loss: 0.4125 - accuracy: 0.8750
Epoch 345/500
1/1 - 0s - loss: 0.4096 - accuracy: 0.8750
Epoch 346/500
1/1 - 0s - loss: 0.4069 - accuracy: 0.8750
Epoch 347/500
1/1 - 0s - loss: 0.4041 - accuracy: 0.8750
Epoch 348/500
1/1 - 0s - loss: 0.4014 - accuracy: 0.8750
Epoch 349/500
1/1 - 0s - loss: 0.3988 - accuracy: 0.8750
Epoch 350/500
1/1 - 0s - loss: 0.3962 - accuracy: 0.8750
Epoch 351/500
1/1 - 0s - loss: 0.3936 - accuracy: 0.8750
Epoch 352/500
1/1 - 0s - loss: 0.3911 - accuracy: 0.8750
Epoch 353/500
1/1 - 0s - loss: 0.3886 - accuracy: 0.8750
Epoch 354/500
1/1 - 0s - loss: 0.3862 - accuracy: 0.8750
Epoch 355/500
1/1 - 0s - loss: 0.3838 - accuracy: 0.8750
Epoch 356/500
1/1 - 0s - loss: 0.3814 - accuracy: 0.8750
Epoch 357/500
1/1 - 0s - loss: 0.3791 - accuracy: 0.8750
Epoch 358/500
1/1 - 0s - loss: 0.3768 - accuracy: 0.8750
Epoch 359/500
1/1 - 0s - loss: 0.3745 - accuracy: 0.8750
Epoch 360/500
1/1 - 0s - loss: 0.3723 - accuracy: 0.8750
Epoch 361/500
1/1 - 0s - loss: 0.3701 - accuracy: 0.8750
Epoch 362/500
1/1 - 0s - loss: 0.3680 - accuracy: 0.8750
Epoch 363/500
1/1 - 0s - loss: 0.3659 - accuracy: 0.8750
Epoch 364/500
1/1 - 0s - loss: 0.3638 - accuracy: 0.8750
Epoch 365/500
1/1 - 0s - loss: 0.3618 - accuracy: 0.8750
Epoch 366/500
1/1 - 0s - loss: 0.3597 - accuracy: 0.8750
Epoch 367/500
1/1 - 0s - loss: 0.3578 - accuracy: 0.8750
Epoch 368/500
1/1 - 0s - loss: 0.3558 - accuracy: 0.8750
Epoch 369/500
1/1 - 0s - loss: 0.3539 - accuracy: 0.8750
Epoch 370/500
1/1 - 0s - loss: 0.3520 - accuracy: 0.8750
Epoch 371/500
1/1 - 0s - loss: 0.3501 - accuracy: 0.8750
Epoch 372/500
1/1 - 0s - loss: 0.3483 - accuracy: 0.8750
Epoch 373/500
1/1 - 0s - loss: 0.3465 - accuracy: 0.8750
Epoch 374/500
1/1 - 0s - loss: 0.3447 - accuracy: 0.8750
Epoch 375/500
1/1 - 0s - loss: 0.3430 - accuracy: 0.8750
Epoch 376/500
1/1 - 0s - loss: 0.3413 - accuracy: 0.8750
Epoch 377/500
1/1 - 0s - loss: 0.3396 - accuracy: 0.8750
Epoch 378/500
1/1 - 0s - loss: 0.3379 - accuracy: 0.8750
Epoch 379/500
1/1 - 0s - loss: 0.3363 - accuracy: 0.8750
Epoch 380/500
1/1 - 0s - loss: 0.3347 - accuracy: 0.8750
Epoch 381/500
1/1 - 0s - loss: 0.3331 - accuracy: 0.8750
Epoch 382/500
1/1 - 0s - loss: 0.3315 - accuracy: 0.8750
Epoch 383/500
1/1 - 0s - loss: 0.3300 - accuracy: 0.8750
Epoch 384/500
1/1 - 0s - loss: 0.3285 - accuracy: 0.8750
Epoch 385/500
1/1 - 0s - loss: 0.3270 - accuracy: 0.8750
Epoch 386/500
1/1 - 0s - loss: 0.3255 - accuracy: 0.8750
Epoch 387/500
1/1 - 0s - loss: 0.3241 - accuracy: 0.8750
Epoch 388/500
1/1 - 0s - loss: 0.3226 - accuracy: 0.8750
Epoch 389/500
1/1 - 0s - loss: 0.3212 - accuracy: 0.8750
Epoch 390/500
1/1 - 0s - loss: 0.3198 - accuracy: 0.8750
Epoch 391/500
1/1 - 0s - loss: 0.3185 - accuracy: 0.8750
Epoch 392/500
1/1 - 0s - loss: 0.3171 - accuracy: 0.8750
Epoch 393/500
1/1 - 0s - loss: 0.3158 - accuracy: 0.8750
Epoch 394/500
1/1 - 0s - loss: 0.3145 - accuracy: 0.8750
Epoch 395/500
1/1 - 0s - loss: 0.3132 - accuracy: 0.8750
Epoch 396/500
1/1 - 0s - loss: 0.3120 - accuracy: 0.8750
Epoch 397/500
1/1 - 0s - loss: 0.3107 - accuracy: 0.8750
Epoch 398/500
1/1 - 0s - loss: 0.3095 - accuracy: 0.8750
Epoch 399/500
1/1 - 0s - loss: 0.3083 - accuracy: 0.8750
Epoch 400/500
1/1 - 0s - loss: 0.3071 - accuracy: 0.8750
Epoch 401/500
1/1 - 0s - loss: 0.3059 - accuracy: 0.8750
Epoch 402/500
1/1 - 0s - loss: 0.3048 - accuracy: 0.8750
Epoch 403/500
1/1 - 0s - loss: 0.3036 - accuracy: 0.8750
Epoch 404/500
1/1 - 0s - loss: 0.3025 - accuracy: 0.8750
Epoch 405/500
1/1 - 0s - loss: 0.3014 - accuracy: 0.8750
Epoch 406/500
1/1 - 0s - loss: 0.3003 - accuracy: 0.8750
Epoch 407/500
1/1 - 0s - loss: 0.2993 - accuracy: 0.8750
Epoch 408/500
1/1 - 0s - loss: 0.2982 - accuracy: 0.8750
Epoch 409/500
1/1 - 0s - loss: 0.2972 - accuracy: 0.8750
Epoch 410/500
1/1 - 0s - loss: 0.2961 - accuracy: 0.8750
Epoch 411/500
1/1 - 0s - loss: 0.2951 - accuracy: 0.8750
Epoch 412/500
1/1 - 0s - loss: 0.2941 - accuracy: 0.8750
Epoch 413/500
1/1 - 0s - loss: 0.2931 - accuracy: 0.8750
Epoch 414/500
1/1 - 0s - loss: 0.2922 - accuracy: 0.8750
Epoch 415/500
1/1 - 0s - loss: 0.2912 - accuracy: 0.8750
Epoch 416/500
1/1 - 0s - loss: 0.2903 - accuracy: 0.8750
Epoch 417/500
1/1 - 0s - loss: 0.2894 - accuracy: 0.8750
Epoch 418/500
1/1 - 0s - loss: 0.2884 - accuracy: 0.8750
Epoch 419/500
1/1 - 0s - loss: 0.2875 - accuracy: 0.8750
Epoch 420/500
1/1 - 0s - loss: 0.2866 - accuracy: 0.8750
Epoch 421/500
1/1 - 0s - loss: 0.2858 - accuracy: 0.8750

Epoch 422/500
1/1 - 0s - loss: 0.2849 - accuracy: 0.8750
Epoch 423/500
1/1 - 0s - loss: 0.2840 - accuracy: 0.8750
Epoch 424/500
1/1 - 0s - loss: 0.2832 - accuracy: 0.8750
Epoch 425/500
1/1 - 0s - loss: 0.2824 - accuracy: 0.8750
Epoch 426/500
1/1 - 0s - loss: 0.2816 - accuracy: 0.8750
Epoch 427/500
1/1 - 0s - loss: 0.2807 - accuracy: 0.8750
Epoch 428/500
1/1 - 0s - loss: 0.2799 - accuracy: 0.8750
Epoch 429/500
1/1 - 0s - loss: 0.2792 - accuracy: 0.8750
Epoch 430/500
1/1 - 0s - loss: 0.2784 - accuracy: 0.8750
Epoch 431/500
1/1 - 0s - loss: 0.2776 - accuracy: 0.8750
Epoch 432/500
1/1 - 0s - loss: 0.2769 - accuracy: 0.8750
Epoch 433/500
1/1 - 0s - loss: 0.2761 - accuracy: 0.8750
Epoch 434/500
1/1 - 0s - loss: 0.2754 - accuracy: 0.8750
Epoch 435/500
1/1 - 0s - loss: 0.2747 - accuracy: 0.8750
Epoch 436/500
1/1 - 0s - loss: 0.2739 - accuracy: 0.8750
Epoch 437/500
1/1 - 0s - loss: 0.2732 - accuracy: 0.8750
Epoch 438/500
1/1 - 0s - loss: 0.2725 - accuracy: 0.8750
Epoch 439/500
1/1 - 0s - loss: 0.2719 - accuracy: 0.8750
Epoch 440/500
1/1 - 0s - loss: 0.2712 - accuracy: 0.8750
Epoch 441/500
1/1 - 0s - loss: 0.2705 - accuracy: 0.8750
Epoch 442/500
1/1 - 0s - loss: 0.2698 - accuracy: 0.8750
Epoch 443/500
1/1 - 0s - loss: 0.2692 - accuracy: 0.8750
Epoch 444/500
1/1 - 0s - loss: 0.2685 - accuracy: 0.8750
Epoch 445/500
1/1 - 0s - loss: 0.2679 - accuracy: 0.8750
Epoch 446/500
1/1 - 0s - loss: 0.2673 - accuracy: 0.8750
Epoch 447/500
1/1 - 0s - loss: 0.2667 - accuracy: 0.8750
Epoch 448/500
1/1 - 0s - loss: 0.2660 - accuracy: 0.8750
Epoch 449/500
1/1 - 0s - loss: 0.2654 - accuracy: 0.8750
Epoch 450/500
1/1 - 0s - loss: 0.2648 - accuracy: 0.8750
Epoch 451/500
1/1 - 0s - loss: 0.2643 - accuracy: 0.8750
Epoch 452/500
1/1 - 0s - loss: 0.2637 - accuracy: 0.8750
Epoch 453/500
1/1 - 0s - loss: 0.2631 - accuracy: 0.8750
Epoch 454/500
1/1 - 0s - loss: 0.2625 - accuracy: 0.8750
Epoch 455/500
1/1 - 0s - loss: 0.2620 - accuracy: 0.8750
Epoch 456/500
1/1 - 0s - loss: 0.2614 - accuracy: 0.8750
Epoch 457/500
1/1 - 0s - loss: 0.2609 - accuracy: 0.8750
Epoch 458/500
1/1 - 0s - loss: 0.2603 - accuracy: 0.8750
Epoch 459/500
1/1 - 0s - loss: 0.2598 - accuracy: 0.8750
Epoch 460/500
1/1 - 0s - loss: 0.2593 - accuracy: 0.8750
Epoch 461/500
1/1 - 0s - loss: 0.2587 - accuracy: 0.8750
Epoch 462/500
1/1 - 0s - loss: 0.2582 - accuracy: 0.8750
Epoch 463/500
1/1 - 0s - loss: 0.2577 - accuracy: 0.8750
Epoch 464/500
1/1 - 0s - loss: 0.2572 - accuracy: 0.8750
Epoch 465/500
1/1 - 0s - loss: 0.2567 - accuracy: 0.8750
Epoch 466/500
1/1 - 0s - loss: 0.2562 - accuracy: 0.8750
Epoch 467/500
1/1 - 0s - loss: 0.2557 - accuracy: 0.8750
Epoch 468/500
1/1 - 0s - loss: 0.2553 - accuracy: 0.8750
Epoch 469/500
1/1 - 0s - loss: 0.2548 - accuracy: 0.8750
Epoch 470/500
1/1 - 0s - loss: 0.2543 - accuracy: 0.8750
Epoch 471/500
1/1 - 0s - loss: 0.2539 - accuracy: 0.8750
Epoch 472/500
1/1 - 0s - loss: 0.2534 - accuracy: 0.8750
Epoch 473/500
1/1 - 0s - loss: 0.2529 - accuracy: 0.8750
Epoch 474/500
1/1 - 0s - loss: 0.2525 - accuracy: 0.8750
Epoch 475/500
1/1 - 0s - loss: 0.2521 - accuracy: 0.8750
Epoch 476/500
1/1 - 0s - loss: 0.2516 - accuracy: 0.8750
Epoch 477/500
1/1 - 0s - loss: 0.2512 - accuracy: 0.8750
Epoch 478/500
1/1 - 0s - loss: 0.2508 - accuracy: 0.8750
Epoch 479/500
1/1 - 0s - loss: 0.2503 - accuracy: 0.8750
Epoch 480/500
1/1 - 0s - loss: 0.2499 - accuracy: 0.8750
Epoch 481/500
1/1 - 0s - loss: 0.2495 - accuracy: 0.8750
Epoch 482/500
1/1 - 0s - loss: 0.2491 - accuracy: 0.8750
Epoch 483/500
1/1 - 0s - loss: 0.2487 - accuracy: 0.8750
Epoch 484/500
1/1 - 0s - loss: 0.2483 - accuracy: 0.8750
Epoch 485/500
1/1 - 0s - loss: 0.2479 - accuracy: 0.8750
Epoch 486/500
1/1 - 0s - loss: 0.2475 - accuracy: 0.8750
Epoch 487/500
1/1 - 0s - loss: 0.2471 - accuracy: 0.8750
Epoch 488/500
1/1 - 0s - loss: 0.2468 - accuracy: 0.8750
Epoch 489/500
1/1 - 0s - loss: 0.2464 - accuracy: 0.8750
Epoch 490/500
1/1 - 0s - loss: 0.2460 - accuracy: 0.8750
Epoch 491/500
1/1 - 0s - loss: 0.2456 - accuracy: 0.8750
Epoch 492/500
1/1 - 0s - loss: 0.2453 - accuracy: 0.8750
Epoch 493/500
1/1 - 0s - loss: 0.2449 - accuracy: 0.8750
Epoch 494/500
1/1 - 0s - loss: 0.2446 - accuracy: 0.8750
Epoch 495/500
1/1 - 0s - loss: 0.2442 - accuracy: 0.8750
Epoch 496/500
1/1 - 0s - loss: 0.2439 - accuracy: 0.8750
Epoch 497/500
1/1 - 0s - loss: 0.2435 - accuracy: 0.8750
Epoch 498/500
1/1 - 0s - loss: 0.2432 - accuracy: 0.8750
Epoch 499/500
1/1 - 0s - loss: 0.2428 - accuracy: 0.8750
Epoch 500/500
1/1 - 0s - loss: 0.2425 - accuracy: 0.8750

<tensorflow.python.keras.callbacks.History at 0x7fed20dd04e0>

plot_model(model)

Failed to import pydot. You must install pydot and graphviz for `pydotprint` to work.

# evaluate
print(generate_seq(model, tokenizer, 'Jack', 6))

WARNING:tensorflow:From <ipython-input-1-21b802b2e424>:18: Sequential.predict_classes (from tensorflow.python.keras.engine.sequential) is deprecated and will be removed after 2021-01-01.
Instructions for updating:
Please use instead:* `np.argmax(model.predict(x), axis=-1)`,   if your model does multi-class classification   (e.g. if it uses a `softmax` last-layer activation).* `(model.predict(x) > 0.5).astype("int32")`,   if your model does binary classification   (e.g. if it uses a `sigmoid` last-layer activation).
Jack and jill came tumbling after his

Line-based Model

from numpy import array
from keras.preprocessing.text import Tokenizer
from keras.utils import to_categorical, plot_model
from keras.preprocessing.sequence import pad_sequences
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from keras.layers import Embedding


# generate a sequence from a language model
def generate_seq(model, tokenizer, max_length, seed_text, n_words):
    in_text = seed_text
    # generate a fixed number of words
    for _ in range(n_words):
        # encode the text as integer
        encoded = tokenizer.texts_to_sequences([in_text])[0]
        # pre-pad sequences to a fixed length
        encoded = pad_sequences([encoded], maxlen=max_length, padding='pre')
        # predict probabilities for each word
        yhat = model.predict_classes(encoded, verbose=0)
        # map predicted word index to word
        out_word = ''
        for word, index in tokenizer.word_index.items():
            if index == yhat:
                out_word = word
                break
        # append to input
        in_text += ' ' + out_word
    return in_text


# source text
data = """ Jack and Jill went up the hill\n
            To fetch a pail of water\n
            Jack fell down and broke his crown\n
            And Jill came tumbling after\n """
# prepare the tokenizer on the source text
tokenizer = Tokenizer(oov_token=1) ## specify the word id for unknown words
tokenizer.fit_on_texts([data])

# determine the vocabulary size
    ## zero index is reserved in keras as the padding token (+1) and one unknown word id 
vocab_size = len(tokenizer.word_index) + 1
print('Vocabulary Size: %d' % vocab_size)
# create line-based sequences
sequences = list()
for line in data.split('\n'):
    encoded = tokenizer.texts_to_sequences([line])[0]
    ## For each line, after converting words into indexes
    ## prepare sequences for training
    ## given a line, w1,w2,w3,w4
    ## create input sequences:
    ## w1,w2
    ## w1,w2,w3
    ## w1,w2,w3,w4
    for i in range(1, len(encoded)):
        sequence = encoded[:i + 1]
        sequences.append(sequence)
print('Total Sequences: %d' % len(sequences))

Vocabulary Size: 23
Total Sequences: 21

# pad input sequences
max_length = max([len(seq) for seq in sequences])
sequences = pad_sequences(sequences, maxlen=max_length, padding='pre')
print('Max Sequence Length: %d' % max_length)

# split into input and output elements
sequences = array(sequences)
X, y = sequences[:,:-1],sequences[:,-1]
y = to_categorical(y, num_classes=vocab_size)
# define model
model = Sequential()
model.add(Embedding(vocab_size, 10, input_length=max_length-1))
model.add(LSTM(50))
model.add(Dense(vocab_size, activation='softmax'))
print(model.summary())
# compile network
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit network
model.fit(X, y, epochs=500, verbose=2)

Show code cell output Hide code cell output

Max Sequence Length: 7
Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_1 (Embedding)      (None, 6, 10)             230       
_________________________________________________________________
lstm_1 (LSTM)                (None, 50)                12200     
_________________________________________________________________
dense_1 (Dense)              (None, 23)                1173      
=================================================================
Total params: 13,603
Trainable params: 13,603
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/500
1/1 - 0s - loss: 3.1348 - accuracy: 0.0952
Epoch 2/500
1/1 - 0s - loss: 3.1332 - accuracy: 0.0952
Epoch 3/500
1/1 - 0s - loss: 3.1316 - accuracy: 0.0952
Epoch 4/500
1/1 - 0s - loss: 3.1300 - accuracy: 0.0952
Epoch 5/500
1/1 - 0s - loss: 3.1283 - accuracy: 0.0952
Epoch 6/500
1/1 - 0s - loss: 3.1266 - accuracy: 0.0952
Epoch 7/500
1/1 - 0s - loss: 3.1249 - accuracy: 0.0952
Epoch 8/500
1/1 - 0s - loss: 3.1231 - accuracy: 0.0952
Epoch 9/500
1/1 - 0s - loss: 3.1212 - accuracy: 0.0952
Epoch 10/500
1/1 - 0s - loss: 3.1192 - accuracy: 0.0952
Epoch 11/500
1/1 - 0s - loss: 3.1171 - accuracy: 0.0952
Epoch 12/500
1/1 - 0s - loss: 3.1150 - accuracy: 0.0952
Epoch 13/500
1/1 - 0s - loss: 3.1127 - accuracy: 0.0952
Epoch 14/500
1/1 - 0s - loss: 3.1103 - accuracy: 0.0952
Epoch 15/500
1/1 - 0s - loss: 3.1078 - accuracy: 0.1429
Epoch 16/500
1/1 - 0s - loss: 3.1051 - accuracy: 0.0952
Epoch 17/500
1/1 - 0s - loss: 3.1023 - accuracy: 0.0952
Epoch 18/500
1/1 - 0s - loss: 3.0993 - accuracy: 0.0952
Epoch 19/500
1/1 - 0s - loss: 3.0960 - accuracy: 0.0952
Epoch 20/500
1/1 - 0s - loss: 3.0926 - accuracy: 0.0952
Epoch 21/500
1/1 - 0s - loss: 3.0889 - accuracy: 0.0952
Epoch 22/500
1/1 - 0s - loss: 3.0849 - accuracy: 0.0952
Epoch 23/500
1/1 - 0s - loss: 3.0807 - accuracy: 0.0952
Epoch 24/500
1/1 - 0s - loss: 3.0761 - accuracy: 0.0952
Epoch 25/500
1/1 - 0s - loss: 3.0712 - accuracy: 0.0952
Epoch 26/500
1/1 - 0s - loss: 3.0659 - accuracy: 0.0952
Epoch 27/500
1/1 - 0s - loss: 3.0602 - accuracy: 0.0952
Epoch 28/500
1/1 - 0s - loss: 3.0541 - accuracy: 0.0952
Epoch 29/500
1/1 - 0s - loss: 3.0474 - accuracy: 0.0952
Epoch 30/500
1/1 - 0s - loss: 3.0403 - accuracy: 0.0952
Epoch 31/500
1/1 - 0s - loss: 3.0326 - accuracy: 0.0952
Epoch 32/500
1/1 - 0s - loss: 3.0243 - accuracy: 0.0952
Epoch 33/500
1/1 - 0s - loss: 3.0154 - accuracy: 0.0952
Epoch 34/500
1/1 - 0s - loss: 3.0058 - accuracy: 0.0952
Epoch 35/500
1/1 - 0s - loss: 2.9956 - accuracy: 0.0952
Epoch 36/500
1/1 - 0s - loss: 2.9848 - accuracy: 0.0952
Epoch 37/500
1/1 - 0s - loss: 2.9735 - accuracy: 0.0952
Epoch 38/500
1/1 - 0s - loss: 2.9617 - accuracy: 0.0952
Epoch 39/500
1/1 - 0s - loss: 2.9495 - accuracy: 0.0952
Epoch 40/500
1/1 - 0s - loss: 2.9373 - accuracy: 0.0952
Epoch 41/500
1/1 - 0s - loss: 2.9251 - accuracy: 0.0952
Epoch 42/500
1/1 - 0s - loss: 2.9134 - accuracy: 0.0952
Epoch 43/500
1/1 - 0s - loss: 2.9024 - accuracy: 0.0952
Epoch 44/500
1/1 - 0s - loss: 2.8921 - accuracy: 0.0952
Epoch 45/500
1/1 - 0s - loss: 2.8827 - accuracy: 0.0952
Epoch 46/500
1/1 - 0s - loss: 2.8737 - accuracy: 0.0952
Epoch 47/500
1/1 - 0s - loss: 2.8647 - accuracy: 0.0952
Epoch 48/500
1/1 - 0s - loss: 2.8553 - accuracy: 0.0952
Epoch 49/500
1/1 - 0s - loss: 2.8452 - accuracy: 0.0952
Epoch 50/500
1/1 - 0s - loss: 2.8343 - accuracy: 0.1429
Epoch 51/500
1/1 - 0s - loss: 2.8229 - accuracy: 0.1429
Epoch 52/500
1/1 - 0s - loss: 2.8109 - accuracy: 0.1429
Epoch 53/500
1/1 - 0s - loss: 2.7986 - accuracy: 0.1429
Epoch 54/500
1/1 - 0s - loss: 2.7860 - accuracy: 0.1429
Epoch 55/500
1/1 - 0s - loss: 2.7733 - accuracy: 0.1429
Epoch 56/500
1/1 - 0s - loss: 2.7602 - accuracy: 0.1429
Epoch 57/500
1/1 - 0s - loss: 2.7467 - accuracy: 0.1429
Epoch 58/500
1/1 - 0s - loss: 2.7325 - accuracy: 0.1429
Epoch 59/500
1/1 - 0s - loss: 2.7177 - accuracy: 0.1429
Epoch 60/500
1/1 - 0s - loss: 2.7021 - accuracy: 0.1429
Epoch 61/500
1/1 - 0s - loss: 2.6858 - accuracy: 0.1429
Epoch 62/500
1/1 - 0s - loss: 2.6687 - accuracy: 0.1429
Epoch 63/500
1/1 - 0s - loss: 2.6509 - accuracy: 0.1429
Epoch 64/500
1/1 - 0s - loss: 2.6324 - accuracy: 0.1429
Epoch 65/500
1/1 - 0s - loss: 2.6133 - accuracy: 0.1429
Epoch 66/500
1/1 - 0s - loss: 2.5934 - accuracy: 0.1429
Epoch 67/500
1/1 - 0s - loss: 2.5726 - accuracy: 0.1429
Epoch 68/500
1/1 - 0s - loss: 2.5510 - accuracy: 0.1905
Epoch 69/500
1/1 - 0s - loss: 2.5286 - accuracy: 0.1905
Epoch 70/500
1/1 - 0s - loss: 2.5055 - accuracy: 0.1905
Epoch 71/500
1/1 - 0s - loss: 2.4820 - accuracy: 0.1905
Epoch 72/500
1/1 - 0s - loss: 2.4581 - accuracy: 0.1905
Epoch 73/500
1/1 - 0s - loss: 2.4337 - accuracy: 0.1905
Epoch 74/500
1/1 - 0s - loss: 2.4089 - accuracy: 0.2381
Epoch 75/500
1/1 - 0s - loss: 2.3836 - accuracy: 0.2381
Epoch 76/500
1/1 - 0s - loss: 2.3580 - accuracy: 0.2381
Epoch 77/500
1/1 - 0s - loss: 2.3324 - accuracy: 0.2381
Epoch 78/500
1/1 - 0s - loss: 2.3068 - accuracy: 0.2381
Epoch 79/500
1/1 - 0s - loss: 2.2813 - accuracy: 0.2381
Epoch 80/500
1/1 - 0s - loss: 2.2560 - accuracy: 0.2381
Epoch 81/500
1/1 - 0s - loss: 2.2309 - accuracy: 0.2381
Epoch 82/500
1/1 - 0s - loss: 2.2063 - accuracy: 0.2857
Epoch 83/500
1/1 - 0s - loss: 2.1820 - accuracy: 0.2857
Epoch 84/500
1/1 - 0s - loss: 2.1580 - accuracy: 0.2857
Epoch 85/500
1/1 - 0s - loss: 2.1341 - accuracy: 0.2857
Epoch 86/500
1/1 - 0s - loss: 2.1106 - accuracy: 0.2857
Epoch 87/500
1/1 - 0s - loss: 2.0873 - accuracy: 0.3333
Epoch 88/500
1/1 - 0s - loss: 2.0641 - accuracy: 0.3333
Epoch 89/500
1/1 - 0s - loss: 2.0410 - accuracy: 0.3333
Epoch 90/500
1/1 - 0s - loss: 2.0180 - accuracy: 0.3333
Epoch 91/500
1/1 - 0s - loss: 1.9947 - accuracy: 0.3810
Epoch 92/500
1/1 - 0s - loss: 1.9715 - accuracy: 0.3810
Epoch 93/500
1/1 - 0s - loss: 1.9484 - accuracy: 0.4286
Epoch 94/500
1/1 - 0s - loss: 1.9252 - accuracy: 0.4762
Epoch 95/500
1/1 - 0s - loss: 1.9020 - accuracy: 0.4762
Epoch 96/500
1/1 - 0s - loss: 1.8787 - accuracy: 0.5238
Epoch 97/500
1/1 - 0s - loss: 1.8552 - accuracy: 0.5238
Epoch 98/500
1/1 - 0s - loss: 1.8318 - accuracy: 0.5238
Epoch 99/500
1/1 - 0s - loss: 1.8082 - accuracy: 0.5238
Epoch 100/500
1/1 - 0s - loss: 1.7847 - accuracy: 0.5238
Epoch 101/500
1/1 - 0s - loss: 1.7615 - accuracy: 0.5238
Epoch 102/500
1/1 - 0s - loss: 1.7384 - accuracy: 0.5238
Epoch 103/500
1/1 - 0s - loss: 1.7156 - accuracy: 0.5238
Epoch 104/500
1/1 - 0s - loss: 1.6929 - accuracy: 0.5238
Epoch 105/500
1/1 - 0s - loss: 1.6706 - accuracy: 0.5238
Epoch 106/500
1/1 - 0s - loss: 1.6485 - accuracy: 0.5238
Epoch 107/500
1/1 - 0s - loss: 1.6264 - accuracy: 0.5238
Epoch 108/500
1/1 - 0s - loss: 1.6043 - accuracy: 0.5238
Epoch 109/500
1/1 - 0s - loss: 1.5821 - accuracy: 0.5714
Epoch 110/500
1/1 - 0s - loss: 1.5598 - accuracy: 0.6190
Epoch 111/500
1/1 - 0s - loss: 1.5374 - accuracy: 0.6190
Epoch 112/500
1/1 - 0s - loss: 1.5150 - accuracy: 0.6190
Epoch 113/500
1/1 - 0s - loss: 1.4927 - accuracy: 0.6190
Epoch 114/500
1/1 - 0s - loss: 1.4704 - accuracy: 0.6190
Epoch 115/500
1/1 - 0s - loss: 1.4484 - accuracy: 0.6190
Epoch 116/500
1/1 - 0s - loss: 1.4266 - accuracy: 0.6190
Epoch 117/500
1/1 - 0s - loss: 1.4050 - accuracy: 0.6190
Epoch 118/500
1/1 - 0s - loss: 1.3836 - accuracy: 0.6190
Epoch 119/500
1/1 - 0s - loss: 1.3622 - accuracy: 0.6190
Epoch 120/500
1/1 - 0s - loss: 1.3410 - accuracy: 0.6190
Epoch 121/500
1/1 - 0s - loss: 1.3199 - accuracy: 0.6190
Epoch 122/500
1/1 - 0s - loss: 1.2990 - accuracy: 0.6190
Epoch 123/500
1/1 - 0s - loss: 1.2781 - accuracy: 0.6190
Epoch 124/500
1/1 - 0s - loss: 1.2575 - accuracy: 0.6190
Epoch 125/500
1/1 - 0s - loss: 1.2371 - accuracy: 0.6190
Epoch 126/500
1/1 - 0s - loss: 1.2173 - accuracy: 0.6667
Epoch 127/500
1/1 - 0s - loss: 1.1982 - accuracy: 0.7143
Epoch 128/500
1/1 - 0s - loss: 1.1801 - accuracy: 0.7143
Epoch 129/500
1/1 - 0s - loss: 1.1607 - accuracy: 0.7143
Epoch 130/500
1/1 - 0s - loss: 1.1413 - accuracy: 0.7143
Epoch 131/500
1/1 - 0s - loss: 1.1239 - accuracy: 0.7143
Epoch 132/500
1/1 - 0s - loss: 1.1067 - accuracy: 0.7143

Epoch 133/500
1/1 - 0s - loss: 1.0887 - accuracy: 0.7143
Epoch 134/500
1/1 - 0s - loss: 1.0718 - accuracy: 0.7143
Epoch 135/500
1/1 - 0s - loss: 1.0559 - accuracy: 0.7143
Epoch 136/500
1/1 - 0s - loss: 1.0394 - accuracy: 0.7619
Epoch 137/500
1/1 - 0s - loss: 1.0234 - accuracy: 0.7619
Epoch 138/500
1/1 - 0s - loss: 1.0086 - accuracy: 0.7619
Epoch 139/500
1/1 - 0s - loss: 0.9936 - accuracy: 0.8095
Epoch 140/500
1/1 - 0s - loss: 0.9787 - accuracy: 0.8095
Epoch 141/500
1/1 - 0s - loss: 0.9649 - accuracy: 0.8095
Epoch 142/500
1/1 - 0s - loss: 0.9513 - accuracy: 0.8095
Epoch 143/500
1/1 - 0s - loss: 0.9375 - accuracy: 0.8095
Epoch 144/500
1/1 - 0s - loss: 0.9245 - accuracy: 0.8095
Epoch 145/500
1/1 - 0s - loss: 0.9121 - accuracy: 0.8095
Epoch 146/500
1/1 - 0s - loss: 0.8995 - accuracy: 0.8095
Epoch 147/500
1/1 - 0s - loss: 0.8871 - accuracy: 0.8095
Epoch 148/500
1/1 - 0s - loss: 0.8755 - accuracy: 0.8095
Epoch 149/500
1/1 - 0s - loss: 0.8641 - accuracy: 0.8095
Epoch 150/500
1/1 - 0s - loss: 0.8526 - accuracy: 0.8095
Epoch 151/500
1/1 - 0s - loss: 0.8414 - accuracy: 0.8095
Epoch 152/500
1/1 - 0s - loss: 0.8307 - accuracy: 0.8095
Epoch 153/500
1/1 - 0s - loss: 0.8203 - accuracy: 0.8095
Epoch 154/500
1/1 - 0s - loss: 0.8099 - accuracy: 0.8095
Epoch 155/500
1/1 - 0s - loss: 0.7996 - accuracy: 0.8095
Epoch 156/500
1/1 - 0s - loss: 0.7896 - accuracy: 0.8095
Epoch 157/500
1/1 - 0s - loss: 0.7801 - accuracy: 0.8095
Epoch 158/500
1/1 - 0s - loss: 0.7706 - accuracy: 0.8095
Epoch 159/500
1/1 - 0s - loss: 0.7613 - accuracy: 0.8095
Epoch 160/500
1/1 - 0s - loss: 0.7521 - accuracy: 0.8095
Epoch 161/500
1/1 - 0s - loss: 0.7431 - accuracy: 0.8095
Epoch 162/500
1/1 - 0s - loss: 0.7344 - accuracy: 0.8095
Epoch 163/500
1/1 - 0s - loss: 0.7259 - accuracy: 0.8095
Epoch 164/500
1/1 - 0s - loss: 0.7177 - accuracy: 0.8095
Epoch 165/500
1/1 - 0s - loss: 0.7096 - accuracy: 0.8095
Epoch 166/500
1/1 - 0s - loss: 0.7018 - accuracy: 0.8095
Epoch 167/500
1/1 - 0s - loss: 0.6941 - accuracy: 0.8095
Epoch 168/500
1/1 - 0s - loss: 0.6865 - accuracy: 0.8095
Epoch 169/500
1/1 - 0s - loss: 0.6791 - accuracy: 0.8095
Epoch 170/500
1/1 - 0s - loss: 0.6718 - accuracy: 0.8095
Epoch 171/500
1/1 - 0s - loss: 0.6647 - accuracy: 0.8095
Epoch 172/500
1/1 - 0s - loss: 0.6577 - accuracy: 0.8095
Epoch 173/500
1/1 - 0s - loss: 0.6510 - accuracy: 0.8095
Epoch 174/500
1/1 - 0s - loss: 0.6444 - accuracy: 0.8095
Epoch 175/500
1/1 - 0s - loss: 0.6380 - accuracy: 0.8095
Epoch 176/500
1/1 - 0s - loss: 0.6317 - accuracy: 0.8095
Epoch 177/500
1/1 - 0s - loss: 0.6256 - accuracy: 0.8095
Epoch 178/500
1/1 - 0s - loss: 0.6197 - accuracy: 0.8095
Epoch 179/500
1/1 - 0s - loss: 0.6140 - accuracy: 0.8095
Epoch 180/500
1/1 - 0s - loss: 0.6086 - accuracy: 0.8095
Epoch 181/500
1/1 - 0s - loss: 0.6035 - accuracy: 0.8095
Epoch 182/500
1/1 - 0s - loss: 0.5985 - accuracy: 0.8095
Epoch 183/500
1/1 - 0s - loss: 0.5926 - accuracy: 0.8095
Epoch 184/500
1/1 - 0s - loss: 0.5864 - accuracy: 0.8095
Epoch 185/500
1/1 - 0s - loss: 0.5810 - accuracy: 0.8095
Epoch 186/500
1/1 - 0s - loss: 0.5766 - accuracy: 0.8095
Epoch 187/500
1/1 - 0s - loss: 0.5719 - accuracy: 0.8095
Epoch 188/500
1/1 - 0s - loss: 0.5665 - accuracy: 0.8095
Epoch 189/500
1/1 - 0s - loss: 0.5613 - accuracy: 0.8095
Epoch 190/500
1/1 - 0s - loss: 0.5570 - accuracy: 0.8095
Epoch 191/500
1/1 - 0s - loss: 0.5526 - accuracy: 0.8095
Epoch 192/500
1/1 - 0s - loss: 0.5477 - accuracy: 0.8095
Epoch 193/500
1/1 - 0s - loss: 0.5430 - accuracy: 0.8095
Epoch 194/500
1/1 - 0s - loss: 0.5388 - accuracy: 0.8095
Epoch 195/500
1/1 - 0s - loss: 0.5347 - accuracy: 0.8571
Epoch 196/500
1/1 - 0s - loss: 0.5301 - accuracy: 0.8571
Epoch 197/500
1/1 - 0s - loss: 0.5258 - accuracy: 0.8571
Epoch 198/500
1/1 - 0s - loss: 0.5218 - accuracy: 0.8571
Epoch 199/500
1/1 - 0s - loss: 0.5178 - accuracy: 0.8571
Epoch 200/500
1/1 - 0s - loss: 0.5135 - accuracy: 0.8571
Epoch 201/500
1/1 - 0s - loss: 0.5095 - accuracy: 0.8571
Epoch 202/500
1/1 - 0s - loss: 0.5057 - accuracy: 0.8571
Epoch 203/500
1/1 - 0s - loss: 0.5018 - accuracy: 0.8571
Epoch 204/500
1/1 - 0s - loss: 0.4978 - accuracy: 0.8571
Epoch 205/500
1/1 - 0s - loss: 0.4940 - accuracy: 0.8571
Epoch 206/500
1/1 - 0s - loss: 0.4904 - accuracy: 0.8571
Epoch 207/500
1/1 - 0s - loss: 0.4866 - accuracy: 0.8571
Epoch 208/500
1/1 - 0s - loss: 0.4829 - accuracy: 0.8571
Epoch 209/500
1/1 - 0s - loss: 0.4793 - accuracy: 0.8571
Epoch 210/500
1/1 - 0s - loss: 0.4758 - accuracy: 0.8571
Epoch 211/500
1/1 - 0s - loss: 0.4722 - accuracy: 0.8571
Epoch 212/500
1/1 - 0s - loss: 0.4687 - accuracy: 0.8571
Epoch 213/500
1/1 - 0s - loss: 0.4653 - accuracy: 0.8571
Epoch 214/500
1/1 - 0s - loss: 0.4619 - accuracy: 0.9048
Epoch 215/500
1/1 - 0s - loss: 0.4585 - accuracy: 0.9048
Epoch 216/500
1/1 - 0s - loss: 0.4551 - accuracy: 0.9048
Epoch 217/500
1/1 - 0s - loss: 0.4518 - accuracy: 0.9048
Epoch 218/500
1/1 - 0s - loss: 0.4485 - accuracy: 0.9048
Epoch 219/500
1/1 - 0s - loss: 0.4452 - accuracy: 0.9048
Epoch 220/500
1/1 - 0s - loss: 0.4420 - accuracy: 0.9048
Epoch 221/500
1/1 - 0s - loss: 0.4388 - accuracy: 0.9048
Epoch 222/500
1/1 - 0s - loss: 0.4356 - accuracy: 0.9048
Epoch 223/500
1/1 - 0s - loss: 0.4325 - accuracy: 0.9048
Epoch 224/500
1/1 - 0s - loss: 0.4293 - accuracy: 0.9048
Epoch 225/500
1/1 - 0s - loss: 0.4262 - accuracy: 0.9048
Epoch 226/500
1/1 - 0s - loss: 0.4232 - accuracy: 0.9048
Epoch 227/500
1/1 - 0s - loss: 0.4201 - accuracy: 0.9048
Epoch 228/500
1/1 - 0s - loss: 0.4170 - accuracy: 0.9048
Epoch 229/500
1/1 - 0s - loss: 0.4140 - accuracy: 0.9048
Epoch 230/500
1/1 - 0s - loss: 0.4111 - accuracy: 0.9048
Epoch 231/500
1/1 - 0s - loss: 0.4081 - accuracy: 0.9524
Epoch 232/500
1/1 - 0s - loss: 0.4051 - accuracy: 0.9524
Epoch 233/500
1/1 - 0s - loss: 0.4022 - accuracy: 0.9524
Epoch 234/500
1/1 - 0s - loss: 0.3993 - accuracy: 0.9524
Epoch 235/500
1/1 - 0s - loss: 0.3964 - accuracy: 0.9524
Epoch 236/500
1/1 - 0s - loss: 0.3935 - accuracy: 0.9524
Epoch 237/500
1/1 - 0s - loss: 0.3906 - accuracy: 0.9524
Epoch 238/500
1/1 - 0s - loss: 0.3878 - accuracy: 0.9524
Epoch 239/500
1/1 - 0s - loss: 0.3850 - accuracy: 0.9524
Epoch 240/500
1/1 - 0s - loss: 0.3822 - accuracy: 0.9524
Epoch 241/500
1/1 - 0s - loss: 0.3794 - accuracy: 0.9524
Epoch 242/500
1/1 - 0s - loss: 0.3766 - accuracy: 0.9524
Epoch 243/500
1/1 - 0s - loss: 0.3738 - accuracy: 0.9524
Epoch 244/500
1/1 - 0s - loss: 0.3711 - accuracy: 0.9524
Epoch 245/500
1/1 - 0s - loss: 0.3684 - accuracy: 0.9524
Epoch 246/500
1/1 - 0s - loss: 0.3657 - accuracy: 0.9524
Epoch 247/500
1/1 - 0s - loss: 0.3630 - accuracy: 0.9524
Epoch 248/500
1/1 - 0s - loss: 0.3603 - accuracy: 0.9524
Epoch 249/500
1/1 - 0s - loss: 0.3576 - accuracy: 0.9524
Epoch 250/500
1/1 - 0s - loss: 0.3550 - accuracy: 0.9524
Epoch 251/500
1/1 - 0s - loss: 0.3523 - accuracy: 0.9524
Epoch 252/500
1/1 - 0s - loss: 0.3497 - accuracy: 0.9524
Epoch 253/500
1/1 - 0s - loss: 0.3471 - accuracy: 0.9524
Epoch 254/500
1/1 - 0s - loss: 0.3445 - accuracy: 0.9524
Epoch 255/500
1/1 - 0s - loss: 0.3419 - accuracy: 0.9524
Epoch 256/500
1/1 - 0s - loss: 0.3393 - accuracy: 0.9524
Epoch 257/500
1/1 - 0s - loss: 0.3368 - accuracy: 0.9524
Epoch 258/500
1/1 - 0s - loss: 0.3342 - accuracy: 0.9524
Epoch 259/500
1/1 - 0s - loss: 0.3317 - accuracy: 0.9524
Epoch 260/500
1/1 - 0s - loss: 0.3292 - accuracy: 0.9524
Epoch 261/500
1/1 - 0s - loss: 0.3267 - accuracy: 0.9524
Epoch 262/500
1/1 - 0s - loss: 0.3242 - accuracy: 0.9524
Epoch 263/500
1/1 - 0s - loss: 0.3217 - accuracy: 0.9524
Epoch 264/500
1/1 - 0s - loss: 0.3193 - accuracy: 0.9524
Epoch 265/500
1/1 - 0s - loss: 0.3169 - accuracy: 0.9524
Epoch 266/500
1/1 - 0s - loss: 0.3144 - accuracy: 0.9524
Epoch 267/500
1/1 - 0s - loss: 0.3120 - accuracy: 0.9524
Epoch 268/500
1/1 - 0s - loss: 0.3097 - accuracy: 0.9524
Epoch 269/500
1/1 - 0s - loss: 0.3073 - accuracy: 0.9524
Epoch 270/500
1/1 - 0s - loss: 0.3049 - accuracy: 0.9524
Epoch 271/500
1/1 - 0s - loss: 0.3026 - accuracy: 0.9524
Epoch 272/500
1/1 - 0s - loss: 0.3003 - accuracy: 0.9524
Epoch 273/500
1/1 - 0s - loss: 0.2980 - accuracy: 0.9524
Epoch 274/500
1/1 - 0s - loss: 0.2957 - accuracy: 0.9524
Epoch 275/500
1/1 - 0s - loss: 0.2934 - accuracy: 0.9524
Epoch 276/500
1/1 - 0s - loss: 0.2912 - accuracy: 0.9524

Epoch 277/500
1/1 - 0s - loss: 0.2890 - accuracy: 0.9524
Epoch 278/500
1/1 - 0s - loss: 0.2867 - accuracy: 0.9524
Epoch 279/500
1/1 - 0s - loss: 0.2845 - accuracy: 0.9524
Epoch 280/500
1/1 - 0s - loss: 0.2824 - accuracy: 0.9524
Epoch 281/500
1/1 - 0s - loss: 0.2802 - accuracy: 0.9524
Epoch 282/500
1/1 - 0s - loss: 0.2781 - accuracy: 0.9524
Epoch 283/500
1/1 - 0s - loss: 0.2759 - accuracy: 0.9524
Epoch 284/500
1/1 - 0s - loss: 0.2738 - accuracy: 0.9524
Epoch 285/500
1/1 - 0s - loss: 0.2717 - accuracy: 0.9524
Epoch 286/500
1/1 - 0s - loss: 0.2697 - accuracy: 0.9524
Epoch 287/500
1/1 - 0s - loss: 0.2676 - accuracy: 0.9524
Epoch 288/500
1/1 - 0s - loss: 0.2656 - accuracy: 0.9524
Epoch 289/500
1/1 - 0s - loss: 0.2636 - accuracy: 0.9524
Epoch 290/500
1/1 - 0s - loss: 0.2616 - accuracy: 0.9524
Epoch 291/500
1/1 - 0s - loss: 0.2596 - accuracy: 0.9524
Epoch 292/500
1/1 - 0s - loss: 0.2576 - accuracy: 0.9524
Epoch 293/500
1/1 - 0s - loss: 0.2557 - accuracy: 0.9524
Epoch 294/500
1/1 - 0s - loss: 0.2538 - accuracy: 0.9524
Epoch 295/500
1/1 - 0s - loss: 0.2519 - accuracy: 0.9524
Epoch 296/500
1/1 - 0s - loss: 0.2500 - accuracy: 0.9524
Epoch 297/500
1/1 - 0s - loss: 0.2482 - accuracy: 0.9524
Epoch 298/500
1/1 - 0s - loss: 0.2463 - accuracy: 0.9524
Epoch 299/500
1/1 - 0s - loss: 0.2445 - accuracy: 0.9524
Epoch 300/500
1/1 - 0s - loss: 0.2427 - accuracy: 0.9524
Epoch 301/500
1/1 - 0s - loss: 0.2409 - accuracy: 0.9524
Epoch 302/500
1/1 - 0s - loss: 0.2392 - accuracy: 0.9524
Epoch 303/500
1/1 - 0s - loss: 0.2374 - accuracy: 0.9524
Epoch 304/500
1/1 - 0s - loss: 0.2357 - accuracy: 0.9524
Epoch 305/500
1/1 - 0s - loss: 0.2340 - accuracy: 0.9524
Epoch 306/500
1/1 - 0s - loss: 0.2323 - accuracy: 0.9524
Epoch 307/500
1/1 - 0s - loss: 0.2307 - accuracy: 0.9524
Epoch 308/500
1/1 - 0s - loss: 0.2290 - accuracy: 0.9524
Epoch 309/500
1/1 - 0s - loss: 0.2274 - accuracy: 0.9524
Epoch 310/500
1/1 - 0s - loss: 0.2258 - accuracy: 0.9524
Epoch 311/500
1/1 - 0s - loss: 0.2242 - accuracy: 0.9524
Epoch 312/500
1/1 - 0s - loss: 0.2227 - accuracy: 0.9524
Epoch 313/500
1/1 - 0s - loss: 0.2211 - accuracy: 0.9524
Epoch 314/500
1/1 - 0s - loss: 0.2196 - accuracy: 0.9524
Epoch 315/500
1/1 - 0s - loss: 0.2180 - accuracy: 0.9524
Epoch 316/500
1/1 - 0s - loss: 0.2165 - accuracy: 0.9524
Epoch 317/500
1/1 - 0s - loss: 0.2151 - accuracy: 0.9524
Epoch 318/500
1/1 - 0s - loss: 0.2136 - accuracy: 0.9524
Epoch 319/500
1/1 - 0s - loss: 0.2121 - accuracy: 0.9524
Epoch 320/500
1/1 - 0s - loss: 0.2107 - accuracy: 0.9524
Epoch 321/500
1/1 - 0s - loss: 0.2093 - accuracy: 0.9524
Epoch 322/500
1/1 - 0s - loss: 0.2079 - accuracy: 0.9524
Epoch 323/500
1/1 - 0s - loss: 0.2065 - accuracy: 0.9524
Epoch 324/500
1/1 - 0s - loss: 0.2052 - accuracy: 0.9524
Epoch 325/500
1/1 - 0s - loss: 0.2038 - accuracy: 0.9524
Epoch 326/500
1/1 - 0s - loss: 0.2025 - accuracy: 0.9524
Epoch 327/500
1/1 - 0s - loss: 0.2011 - accuracy: 0.9524
Epoch 328/500
1/1 - 0s - loss: 0.1998 - accuracy: 0.9524
Epoch 329/500
1/1 - 0s - loss: 0.1986 - accuracy: 0.9524
Epoch 330/500
1/1 - 0s - loss: 0.1973 - accuracy: 0.9524
Epoch 331/500
1/1 - 0s - loss: 0.1960 - accuracy: 0.9524
Epoch 332/500
1/1 - 0s - loss: 0.1948 - accuracy: 0.9524
Epoch 333/500
1/1 - 0s - loss: 0.1936 - accuracy: 0.9524
Epoch 334/500
1/1 - 0s - loss: 0.1924 - accuracy: 0.9524
Epoch 335/500
1/1 - 0s - loss: 0.1912 - accuracy: 0.9524
Epoch 336/500
1/1 - 0s - loss: 0.1900 - accuracy: 0.9524
Epoch 337/500
1/1 - 0s - loss: 0.1888 - accuracy: 0.9524
Epoch 338/500
1/1 - 0s - loss: 0.1877 - accuracy: 0.9524
Epoch 339/500
1/1 - 0s - loss: 0.1865 - accuracy: 0.9524
Epoch 340/500
1/1 - 0s - loss: 0.1854 - accuracy: 0.9524
Epoch 341/500
1/1 - 0s - loss: 0.1843 - accuracy: 0.9524
Epoch 342/500
1/1 - 0s - loss: 0.1832 - accuracy: 0.9524
Epoch 343/500
1/1 - 0s - loss: 0.1821 - accuracy: 0.9524
Epoch 344/500
1/1 - 0s - loss: 0.1810 - accuracy: 0.9524
Epoch 345/500
1/1 - 0s - loss: 0.1800 - accuracy: 0.9524
Epoch 346/500
1/1 - 0s - loss: 0.1789 - accuracy: 0.9524
Epoch 347/500
1/1 - 0s - loss: 0.1779 - accuracy: 0.9524
Epoch 348/500
1/1 - 0s - loss: 0.1769 - accuracy: 0.9524
Epoch 349/500
1/1 - 0s - loss: 0.1759 - accuracy: 0.9524
Epoch 350/500
1/1 - 0s - loss: 0.1749 - accuracy: 0.9524
Epoch 351/500
1/1 - 0s - loss: 0.1739 - accuracy: 0.9524
Epoch 352/500
1/1 - 0s - loss: 0.1729 - accuracy: 0.9524
Epoch 353/500
1/1 - 0s - loss: 0.1720 - accuracy: 0.9524
Epoch 354/500
1/1 - 0s - loss: 0.1710 - accuracy: 0.9524
Epoch 355/500
1/1 - 0s - loss: 0.1701 - accuracy: 0.9524
Epoch 356/500
1/1 - 0s - loss: 0.1692 - accuracy: 0.9524
Epoch 357/500
1/1 - 0s - loss: 0.1682 - accuracy: 0.9524
Epoch 358/500
1/1 - 0s - loss: 0.1673 - accuracy: 0.9524
Epoch 359/500
1/1 - 0s - loss: 0.1664 - accuracy: 0.9524
Epoch 360/500
1/1 - 0s - loss: 0.1656 - accuracy: 0.9524
Epoch 361/500
1/1 - 0s - loss: 0.1647 - accuracy: 0.9524
Epoch 362/500
1/1 - 0s - loss: 0.1638 - accuracy: 0.9524
Epoch 363/500
1/1 - 0s - loss: 0.1630 - accuracy: 0.9524
Epoch 364/500
1/1 - 0s - loss: 0.1621 - accuracy: 0.9524
Epoch 365/500
1/1 - 0s - loss: 0.1613 - accuracy: 0.9524
Epoch 366/500
1/1 - 0s - loss: 0.1605 - accuracy: 0.9524
Epoch 367/500
1/1 - 0s - loss: 0.1597 - accuracy: 0.9524
Epoch 368/500
1/1 - 0s - loss: 0.1589 - accuracy: 0.9524
Epoch 369/500
1/1 - 0s - loss: 0.1581 - accuracy: 0.9524
Epoch 370/500
1/1 - 0s - loss: 0.1573 - accuracy: 0.9524
Epoch 371/500
1/1 - 0s - loss: 0.1565 - accuracy: 0.9524
Epoch 372/500
1/1 - 0s - loss: 0.1558 - accuracy: 0.9524
Epoch 373/500
1/1 - 0s - loss: 0.1550 - accuracy: 0.9524
Epoch 374/500
1/1 - 0s - loss: 0.1542 - accuracy: 0.9524
Epoch 375/500
1/1 - 0s - loss: 0.1535 - accuracy: 0.9524
Epoch 376/500
1/1 - 0s - loss: 0.1528 - accuracy: 0.9524
Epoch 377/500
1/1 - 0s - loss: 0.1521 - accuracy: 0.9524
Epoch 378/500
1/1 - 0s - loss: 0.1513 - accuracy: 0.9524
Epoch 379/500
1/1 - 0s - loss: 0.1506 - accuracy: 0.9524
Epoch 380/500
1/1 - 0s - loss: 0.1499 - accuracy: 0.9524
Epoch 381/500
1/1 - 0s - loss: 0.1493 - accuracy: 0.9524
Epoch 382/500
1/1 - 0s - loss: 0.1486 - accuracy: 0.9524
Epoch 383/500
1/1 - 0s - loss: 0.1479 - accuracy: 0.9524
Epoch 384/500
1/1 - 0s - loss: 0.1472 - accuracy: 0.9524
Epoch 385/500
1/1 - 0s - loss: 0.1466 - accuracy: 0.9524
Epoch 386/500
1/1 - 0s - loss: 0.1459 - accuracy: 0.9524
Epoch 387/500
1/1 - 0s - loss: 0.1453 - accuracy: 0.9524
Epoch 388/500
1/1 - 0s - loss: 0.1446 - accuracy: 0.9524
Epoch 389/500
1/1 - 0s - loss: 0.1440 - accuracy: 0.9524
Epoch 390/500
1/1 - 0s - loss: 0.1434 - accuracy: 0.9524
Epoch 391/500
1/1 - 0s - loss: 0.1428 - accuracy: 0.9524
Epoch 392/500
1/1 - 0s - loss: 0.1422 - accuracy: 0.9524
Epoch 393/500
1/1 - 0s - loss: 0.1416 - accuracy: 0.9524
Epoch 394/500
1/1 - 0s - loss: 0.1410 - accuracy: 0.9524
Epoch 395/500
1/1 - 0s - loss: 0.1404 - accuracy: 0.9524
Epoch 396/500
1/1 - 0s - loss: 0.1398 - accuracy: 0.9524
Epoch 397/500
1/1 - 0s - loss: 0.1392 - accuracy: 0.9524
Epoch 398/500
1/1 - 0s - loss: 0.1387 - accuracy: 0.9524
Epoch 399/500
1/1 - 0s - loss: 0.1381 - accuracy: 0.9524
Epoch 400/500
1/1 - 0s - loss: 0.1375 - accuracy: 0.9524
Epoch 401/500
1/1 - 0s - loss: 0.1370 - accuracy: 0.9524
Epoch 402/500
1/1 - 0s - loss: 0.1364 - accuracy: 0.9524
Epoch 403/500
1/1 - 0s - loss: 0.1359 - accuracy: 0.9524
Epoch 404/500
1/1 - 0s - loss: 0.1354 - accuracy: 0.9524
Epoch 405/500
1/1 - 0s - loss: 0.1348 - accuracy: 0.9524
Epoch 406/500
1/1 - 0s - loss: 0.1343 - accuracy: 0.9524
Epoch 407/500
1/1 - 0s - loss: 0.1338 - accuracy: 0.9524
Epoch 408/500
1/1 - 0s - loss: 0.1333 - accuracy: 0.9524
Epoch 409/500
1/1 - 0s - loss: 0.1328 - accuracy: 0.9524
Epoch 410/500
1/1 - 0s - loss: 0.1323 - accuracy: 0.9524
Epoch 411/500
1/1 - 0s - loss: 0.1318 - accuracy: 0.9524
Epoch 412/500
1/1 - 0s - loss: 0.1313 - accuracy: 0.9524
Epoch 413/500
1/1 - 0s - loss: 0.1308 - accuracy: 0.9524
Epoch 414/500
1/1 - 0s - loss: 0.1303 - accuracy: 0.9524
Epoch 415/500
1/1 - 0s - loss: 0.1299 - accuracy: 0.9524
Epoch 416/500
1/1 - 0s - loss: 0.1294 - accuracy: 0.9524
Epoch 417/500
1/1 - 0s - loss: 0.1289 - accuracy: 0.9524
Epoch 418/500
1/1 - 0s - loss: 0.1285 - accuracy: 0.9524
Epoch 419/500
1/1 - 0s - loss: 0.1280 - accuracy: 0.9524
Epoch 420/500
1/1 - 0s - loss: 0.1276 - accuracy: 0.9524

Epoch 421/500
1/1 - 0s - loss: 0.1271 - accuracy: 0.9524
Epoch 422/500
1/1 - 0s - loss: 0.1267 - accuracy: 0.9524
Epoch 423/500
1/1 - 0s - loss: 0.1263 - accuracy: 0.9524
Epoch 424/500
1/1 - 0s - loss: 0.1258 - accuracy: 0.9524
Epoch 425/500
1/1 - 0s - loss: 0.1254 - accuracy: 0.9524
Epoch 426/500
1/1 - 0s - loss: 0.1250 - accuracy: 0.9524
Epoch 427/500
1/1 - 0s - loss: 0.1246 - accuracy: 0.9524
Epoch 428/500
1/1 - 0s - loss: 0.1241 - accuracy: 0.9524
Epoch 429/500
1/1 - 0s - loss: 0.1237 - accuracy: 0.9524
Epoch 430/500
1/1 - 0s - loss: 0.1233 - accuracy: 0.9524
Epoch 431/500
1/1 - 0s - loss: 0.1229 - accuracy: 0.9524
Epoch 432/500
1/1 - 0s - loss: 0.1225 - accuracy: 0.9524
Epoch 433/500
1/1 - 0s - loss: 0.1221 - accuracy: 0.9524
Epoch 434/500
1/1 - 0s - loss: 0.1217 - accuracy: 0.9524
Epoch 435/500
1/1 - 0s - loss: 0.1214 - accuracy: 0.9524
Epoch 436/500
1/1 - 0s - loss: 0.1210 - accuracy: 0.9524
Epoch 437/500
1/1 - 0s - loss: 0.1206 - accuracy: 0.9524
Epoch 438/500
1/1 - 0s - loss: 0.1202 - accuracy: 0.9524
Epoch 439/500
1/1 - 0s - loss: 0.1198 - accuracy: 0.9524
Epoch 440/500
1/1 - 0s - loss: 0.1195 - accuracy: 0.9524
Epoch 441/500
1/1 - 0s - loss: 0.1191 - accuracy: 0.9524
Epoch 442/500
1/1 - 0s - loss: 0.1188 - accuracy: 0.9524
Epoch 443/500
1/1 - 0s - loss: 0.1184 - accuracy: 0.9524
Epoch 444/500
1/1 - 0s - loss: 0.1180 - accuracy: 0.9524
Epoch 445/500
1/1 - 0s - loss: 0.1177 - accuracy: 0.9524
Epoch 446/500
1/1 - 0s - loss: 0.1173 - accuracy: 0.9524
Epoch 447/500
1/1 - 0s - loss: 0.1170 - accuracy: 0.9524
Epoch 448/500
1/1 - 0s - loss: 0.1167 - accuracy: 0.9524
Epoch 449/500
1/1 - 0s - loss: 0.1163 - accuracy: 0.9524
Epoch 450/500
1/1 - 0s - loss: 0.1160 - accuracy: 0.9524
Epoch 451/500
1/1 - 0s - loss: 0.1157 - accuracy: 0.9524
Epoch 452/500
1/1 - 0s - loss: 0.1153 - accuracy: 0.9524
Epoch 453/500
1/1 - 0s - loss: 0.1150 - accuracy: 0.9524
Epoch 454/500
1/1 - 0s - loss: 0.1147 - accuracy: 0.9524
Epoch 455/500
1/1 - 0s - loss: 0.1144 - accuracy: 0.9524
Epoch 456/500
1/1 - 0s - loss: 0.1140 - accuracy: 0.9524
Epoch 457/500
1/1 - 0s - loss: 0.1137 - accuracy: 0.9524
Epoch 458/500
1/1 - 0s - loss: 0.1134 - accuracy: 0.9524
Epoch 459/500
1/1 - 0s - loss: 0.1131 - accuracy: 0.9524
Epoch 460/500
1/1 - 0s - loss: 0.1128 - accuracy: 0.9524
Epoch 461/500
1/1 - 0s - loss: 0.1125 - accuracy: 0.9524
Epoch 462/500
1/1 - 0s - loss: 0.1122 - accuracy: 0.9524
Epoch 463/500
1/1 - 0s - loss: 0.1119 - accuracy: 0.9524
Epoch 464/500
1/1 - 0s - loss: 0.1116 - accuracy: 0.9524
Epoch 465/500
1/1 - 0s - loss: 0.1113 - accuracy: 0.9524
Epoch 466/500
1/1 - 0s - loss: 0.1110 - accuracy: 0.9524
Epoch 467/500
1/1 - 0s - loss: 0.1108 - accuracy: 0.9524
Epoch 468/500
1/1 - 0s - loss: 0.1105 - accuracy: 0.9524
Epoch 469/500
1/1 - 0s - loss: 0.1102 - accuracy: 0.9524
Epoch 470/500
1/1 - 0s - loss: 0.1099 - accuracy: 0.9524
Epoch 471/500
1/1 - 0s - loss: 0.1097 - accuracy: 0.9524
Epoch 472/500
1/1 - 0s - loss: 0.1094 - accuracy: 0.9524
Epoch 473/500
1/1 - 0s - loss: 0.1091 - accuracy: 0.9524
Epoch 474/500
1/1 - 0s - loss: 0.1088 - accuracy: 0.9524
Epoch 475/500
1/1 - 0s - loss: 0.1086 - accuracy: 0.9524
Epoch 476/500
1/1 - 0s - loss: 0.1083 - accuracy: 0.9524
Epoch 477/500
1/1 - 0s - loss: 0.1081 - accuracy: 0.9524
Epoch 478/500
1/1 - 0s - loss: 0.1078 - accuracy: 0.9524
Epoch 479/500
1/1 - 0s - loss: 0.1075 - accuracy: 0.9524
Epoch 480/500
1/1 - 0s - loss: 0.1073 - accuracy: 0.9524
Epoch 481/500
1/1 - 0s - loss: 0.1070 - accuracy: 0.9524
Epoch 482/500
1/1 - 0s - loss: 0.1068 - accuracy: 0.9524
Epoch 483/500
1/1 - 0s - loss: 0.1066 - accuracy: 0.9524
Epoch 484/500
1/1 - 0s - loss: 0.1063 - accuracy: 0.9524
Epoch 485/500
1/1 - 0s - loss: 0.1061 - accuracy: 0.9524
Epoch 486/500
1/1 - 0s - loss: 0.1058 - accuracy: 0.9524
Epoch 487/500
1/1 - 0s - loss: 0.1056 - accuracy: 0.9524
Epoch 488/500
1/1 - 0s - loss: 0.1054 - accuracy: 0.9524
Epoch 489/500
1/1 - 0s - loss: 0.1051 - accuracy: 0.9524
Epoch 490/500
1/1 - 0s - loss: 0.1049 - accuracy: 0.9524
Epoch 491/500
1/1 - 0s - loss: 0.1047 - accuracy: 0.9524
Epoch 492/500
1/1 - 0s - loss: 0.1044 - accuracy: 0.9524
Epoch 493/500
1/1 - 0s - loss: 0.1042 - accuracy: 0.9524
Epoch 494/500
1/1 - 0s - loss: 0.1040 - accuracy: 0.9524
Epoch 495/500
1/1 - 0s - loss: 0.1038 - accuracy: 0.9524
Epoch 496/500
1/1 - 0s - loss: 0.1036 - accuracy: 0.9524
Epoch 497/500
1/1 - 0s - loss: 0.1033 - accuracy: 0.9524
Epoch 498/500
1/1 - 0s - loss: 0.1031 - accuracy: 0.9524
Epoch 499/500
1/1 - 0s - loss: 0.1029 - accuracy: 0.9524
Epoch 500/500
1/1 - 0s - loss: 0.1027 - accuracy: 0.9524

<tensorflow.python.keras.callbacks.History at 0x7fed522314a8>

plot_model(model)

Failed to import pydot. You must install pydot and graphviz for `pydotprint` to work.

# evaluate model
print(generate_seq(model, tokenizer, max_length-1, 'Jack', 4))
print(generate_seq(model, tokenizer, max_length-1, 'Jill', 4))

Jack fell down and broke
Jill jill came tumbling after

Trigram Model

from numpy import array
from keras.preprocessing.text import Tokenizer
from keras.utils import to_categorical, plot_model
from keras.preprocessing.sequence import pad_sequences
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from keras.layers import Embedding

# generate a sequence from a language model
def generate_seq(model, tokenizer, max_length, seed_text, n_words):
	in_text = seed_text
	# generate a fixed number of words
	for _ in range(n_words):
		# encode the text as integer
		encoded = tokenizer.texts_to_sequences([in_text])[0]
		# pre-pad sequences to a fixed length
		encoded = pad_sequences([encoded], maxlen=max_length, padding='pre')
		# predict probabilities for each word
		yhat = model.predict_classes(encoded, verbose=0)
		# map predicted word index to word
		out_word = ''
		for word, index in tokenizer.word_index.items():
			if index == yhat:
				out_word = word
				break
		# append to input
		in_text += ' ' + out_word
	return in_text

# source text
data = """ Jack and Jill went up the hill\n
		To fetch a pail of water\n
		Jack fell down and broke his crown\n
		And Jill came tumbling after\n """
# integer encode sequences of words
tokenizer = Tokenizer()
tokenizer.fit_on_texts([data])
encoded = tokenizer.texts_to_sequences([data])[0]
# retrieve vocabulary size
vocab_size = len(tokenizer.word_index) + 1
print('Vocabulary Size: %d' % vocab_size)
# encode 2 words -> 1 word
sequences = list()
for i in range(2, len(encoded)):
	sequence = encoded[i-2:i+1]
	sequences.append(sequence)
print('Total Sequences: %d' % len(sequences))
# pad sequences
max_length = max([len(seq) for seq in sequences])
sequences = pad_sequences(sequences, maxlen=max_length, padding='pre')
print('Max Sequence Length: %d' % max_length)
# split into input and output elements
sequences = array(sequences)
X, y = sequences[:,:-1],sequences[:,-1]
y = to_categorical(y, num_classes=vocab_size)
# define model
model = Sequential()
model.add(Embedding(vocab_size, 10, input_length=max_length-1))
model.add(LSTM(50))
model.add(Dense(vocab_size, activation='softmax'))
print(model.summary())
# compile network
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit network
model.fit(X, y, epochs=500, verbose=2)

Show code cell output Hide code cell output

Vocabulary Size: 22
Total Sequences: 23
Max Sequence Length: 3
Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_2 (Embedding)      (None, 2, 10)             220       
_________________________________________________________________
lstm_2 (LSTM)                (None, 50)                12200     
_________________________________________________________________
dense_2 (Dense)              (None, 22)                1122      
=================================================================
Total params: 13,542
Trainable params: 13,542
Non-trainable params: 0
_________________________________________________________________
None
Epoch 1/500
1/1 - 0s - loss: 3.0909 - accuracy: 0.0870
Epoch 2/500
1/1 - 0s - loss: 3.0901 - accuracy: 0.0870
Epoch 3/500
1/1 - 0s - loss: 3.0892 - accuracy: 0.0870
Epoch 4/500
1/1 - 0s - loss: 3.0884 - accuracy: 0.0870
Epoch 5/500
1/1 - 0s - loss: 3.0875 - accuracy: 0.0870
Epoch 6/500
1/1 - 0s - loss: 3.0867 - accuracy: 0.0870
Epoch 7/500
1/1 - 0s - loss: 3.0858 - accuracy: 0.0870
Epoch 8/500
1/1 - 0s - loss: 3.0849 - accuracy: 0.0870
Epoch 9/500
1/1 - 0s - loss: 3.0840 - accuracy: 0.0870
Epoch 10/500
1/1 - 0s - loss: 3.0831 - accuracy: 0.0870
Epoch 11/500
1/1 - 0s - loss: 3.0822 - accuracy: 0.0870
Epoch 12/500
1/1 - 0s - loss: 3.0813 - accuracy: 0.0870
Epoch 13/500
1/1 - 0s - loss: 3.0803 - accuracy: 0.0870
Epoch 14/500
1/1 - 0s - loss: 3.0793 - accuracy: 0.0870
Epoch 15/500
1/1 - 0s - loss: 3.0783 - accuracy: 0.0870
Epoch 16/500
1/1 - 0s - loss: 3.0772 - accuracy: 0.0870
Epoch 17/500
1/1 - 0s - loss: 3.0762 - accuracy: 0.0870
Epoch 18/500
1/1 - 0s - loss: 3.0751 - accuracy: 0.0870
Epoch 19/500
1/1 - 0s - loss: 3.0739 - accuracy: 0.0870
Epoch 20/500
1/1 - 0s - loss: 3.0728 - accuracy: 0.0870
Epoch 21/500
1/1 - 0s - loss: 3.0715 - accuracy: 0.0870
Epoch 22/500
1/1 - 0s - loss: 3.0703 - accuracy: 0.0870
Epoch 23/500
1/1 - 0s - loss: 3.0690 - accuracy: 0.0870
Epoch 24/500
1/1 - 0s - loss: 3.0676 - accuracy: 0.0870
Epoch 25/500
1/1 - 0s - loss: 3.0662 - accuracy: 0.0870
Epoch 26/500
1/1 - 0s - loss: 3.0648 - accuracy: 0.0870
Epoch 27/500
1/1 - 0s - loss: 3.0633 - accuracy: 0.0870
Epoch 28/500
1/1 - 0s - loss: 3.0617 - accuracy: 0.0870
Epoch 29/500
1/1 - 0s - loss: 3.0601 - accuracy: 0.0870
Epoch 30/500
1/1 - 0s - loss: 3.0584 - accuracy: 0.0870
Epoch 31/500
1/1 - 0s - loss: 3.0567 - accuracy: 0.0870
Epoch 32/500
1/1 - 0s - loss: 3.0549 - accuracy: 0.0870
Epoch 33/500
1/1 - 0s - loss: 3.0530 - accuracy: 0.0870
Epoch 34/500
1/1 - 0s - loss: 3.0510 - accuracy: 0.0870
Epoch 35/500
1/1 - 0s - loss: 3.0490 - accuracy: 0.0870
Epoch 36/500
1/1 - 0s - loss: 3.0469 - accuracy: 0.0870
Epoch 37/500
1/1 - 0s - loss: 3.0446 - accuracy: 0.0870
Epoch 38/500
1/1 - 0s - loss: 3.0424 - accuracy: 0.0870
Epoch 39/500
1/1 - 0s - loss: 3.0400 - accuracy: 0.0870
Epoch 40/500
1/1 - 0s - loss: 3.0375 - accuracy: 0.0870
Epoch 41/500
1/1 - 0s - loss: 3.0349 - accuracy: 0.0870
Epoch 42/500
1/1 - 0s - loss: 3.0322 - accuracy: 0.0870
Epoch 43/500
1/1 - 0s - loss: 3.0294 - accuracy: 0.0870
Epoch 44/500
1/1 - 0s - loss: 3.0264 - accuracy: 0.0870
Epoch 45/500
1/1 - 0s - loss: 3.0234 - accuracy: 0.0870
Epoch 46/500
1/1 - 0s - loss: 3.0202 - accuracy: 0.0870
Epoch 47/500
1/1 - 0s - loss: 3.0169 - accuracy: 0.0870
Epoch 48/500
1/1 - 0s - loss: 3.0134 - accuracy: 0.0870
Epoch 49/500
1/1 - 0s - loss: 3.0099 - accuracy: 0.0870
Epoch 50/500
1/1 - 0s - loss: 3.0061 - accuracy: 0.0870
Epoch 51/500
1/1 - 0s - loss: 3.0022 - accuracy: 0.0870
Epoch 52/500
1/1 - 0s - loss: 2.9982 - accuracy: 0.0870
Epoch 53/500
1/1 - 0s - loss: 2.9939 - accuracy: 0.0870
Epoch 54/500
1/1 - 0s - loss: 2.9895 - accuracy: 0.0870
Epoch 55/500
1/1 - 0s - loss: 2.9850 - accuracy: 0.0870
Epoch 56/500
1/1 - 0s - loss: 2.9802 - accuracy: 0.0870
Epoch 57/500
1/1 - 0s - loss: 2.9752 - accuracy: 0.0870
Epoch 58/500
1/1 - 0s - loss: 2.9700 - accuracy: 0.0870
Epoch 59/500
1/1 - 0s - loss: 2.9646 - accuracy: 0.0870
Epoch 60/500
1/1 - 0s - loss: 2.9590 - accuracy: 0.0870
Epoch 61/500
1/1 - 0s - loss: 2.9532 - accuracy: 0.0870
Epoch 62/500
1/1 - 0s - loss: 2.9471 - accuracy: 0.0870
Epoch 63/500
1/1 - 0s - loss: 2.9408 - accuracy: 0.0870
Epoch 64/500
1/1 - 0s - loss: 2.9342 - accuracy: 0.0870
Epoch 65/500
1/1 - 0s - loss: 2.9274 - accuracy: 0.0870
Epoch 66/500
1/1 - 0s - loss: 2.9202 - accuracy: 0.0870
Epoch 67/500
1/1 - 0s - loss: 2.9128 - accuracy: 0.0870
Epoch 68/500
1/1 - 0s - loss: 2.9052 - accuracy: 0.0870
Epoch 69/500
1/1 - 0s - loss: 2.8972 - accuracy: 0.0870
Epoch 70/500
1/1 - 0s - loss: 2.8888 - accuracy: 0.0870
Epoch 71/500
1/1 - 0s - loss: 2.8802 - accuracy: 0.0870
Epoch 72/500
1/1 - 0s - loss: 2.8713 - accuracy: 0.0870
Epoch 73/500
1/1 - 0s - loss: 2.8620 - accuracy: 0.0870
Epoch 74/500
1/1 - 0s - loss: 2.8523 - accuracy: 0.0870
Epoch 75/500
1/1 - 0s - loss: 2.8423 - accuracy: 0.0870
Epoch 76/500
1/1 - 0s - loss: 2.8319 - accuracy: 0.0870
Epoch 77/500
1/1 - 0s - loss: 2.8211 - accuracy: 0.0870
Epoch 78/500
1/1 - 0s - loss: 2.8100 - accuracy: 0.0870
Epoch 79/500
1/1 - 0s - loss: 2.7984 - accuracy: 0.0870
Epoch 80/500
1/1 - 0s - loss: 2.7865 - accuracy: 0.0870
Epoch 81/500
1/1 - 0s - loss: 2.7741 - accuracy: 0.0870
Epoch 82/500
1/1 - 0s - loss: 2.7613 - accuracy: 0.0870
Epoch 83/500
1/1 - 0s - loss: 2.7480 - accuracy: 0.0870
Epoch 84/500
1/1 - 0s - loss: 2.7344 - accuracy: 0.0870
Epoch 85/500
1/1 - 0s - loss: 2.7202 - accuracy: 0.0870
Epoch 86/500
1/1 - 0s - loss: 2.7057 - accuracy: 0.0870
Epoch 87/500
1/1 - 0s - loss: 2.6907 - accuracy: 0.1304
Epoch 88/500
1/1 - 0s - loss: 2.6752 - accuracy: 0.1739
Epoch 89/500
1/1 - 0s - loss: 2.6592 - accuracy: 0.1739
Epoch 90/500
1/1 - 0s - loss: 2.6428 - accuracy: 0.1739
Epoch 91/500
1/1 - 0s - loss: 2.6259 - accuracy: 0.1739
Epoch 92/500
1/1 - 0s - loss: 2.6085 - accuracy: 0.1739
Epoch 93/500
1/1 - 0s - loss: 2.5906 - accuracy: 0.1739
Epoch 94/500
1/1 - 0s - loss: 2.5723 - accuracy: 0.1739
Epoch 95/500
1/1 - 0s - loss: 2.5535 - accuracy: 0.1739
Epoch 96/500
1/1 - 0s - loss: 2.5342 - accuracy: 0.1739
Epoch 97/500
1/1 - 0s - loss: 2.5144 - accuracy: 0.1739
Epoch 98/500
1/1 - 0s - loss: 2.4942 - accuracy: 0.1739
Epoch 99/500
1/1 - 0s - loss: 2.4734 - accuracy: 0.2174
Epoch 100/500
1/1 - 0s - loss: 2.4522 - accuracy: 0.2174
Epoch 101/500
1/1 - 0s - loss: 2.4306 - accuracy: 0.2174
Epoch 102/500
1/1 - 0s - loss: 2.4085 - accuracy: 0.2174
Epoch 103/500
1/1 - 0s - loss: 2.3859 - accuracy: 0.2174
Epoch 104/500
1/1 - 0s - loss: 2.3629 - accuracy: 0.2609
Epoch 105/500
1/1 - 0s - loss: 2.3394 - accuracy: 0.2609
Epoch 106/500
1/1 - 0s - loss: 2.3156 - accuracy: 0.2609
Epoch 107/500
1/1 - 0s - loss: 2.2913 - accuracy: 0.2609
Epoch 108/500
1/1 - 0s - loss: 2.2667 - accuracy: 0.2609
Epoch 109/500
1/1 - 0s - loss: 2.2416 - accuracy: 0.2609
Epoch 110/500
1/1 - 0s - loss: 2.2162 - accuracy: 0.2609
Epoch 111/500
1/1 - 0s - loss: 2.1905 - accuracy: 0.3478
Epoch 112/500
1/1 - 0s - loss: 2.1644 - accuracy: 0.3478
Epoch 113/500
1/1 - 0s - loss: 2.1381 - accuracy: 0.4783
Epoch 114/500
1/1 - 0s - loss: 2.1114 - accuracy: 0.5217
Epoch 115/500
1/1 - 0s - loss: 2.0845 - accuracy: 0.5217
Epoch 116/500
1/1 - 0s - loss: 2.0573 - accuracy: 0.5217
Epoch 117/500
1/1 - 0s - loss: 2.0299 - accuracy: 0.5652
Epoch 118/500
1/1 - 0s - loss: 2.0023 - accuracy: 0.5652
Epoch 119/500
1/1 - 0s - loss: 1.9745 - accuracy: 0.5652
Epoch 120/500
1/1 - 0s - loss: 1.9465 - accuracy: 0.5652
Epoch 121/500
1/1 - 0s - loss: 1.9184 - accuracy: 0.6522
Epoch 122/500
1/1 - 0s - loss: 1.8902 - accuracy: 0.6957
Epoch 123/500
1/1 - 0s - loss: 1.8618 - accuracy: 0.6957
Epoch 124/500
1/1 - 0s - loss: 1.8334 - accuracy: 0.6957
Epoch 125/500
1/1 - 0s - loss: 1.8048 - accuracy: 0.6957
Epoch 126/500
1/1 - 0s - loss: 1.7763 - accuracy: 0.6957
Epoch 127/500
1/1 - 0s - loss: 1.7477 - accuracy: 0.6957
Epoch 128/500
1/1 - 0s - loss: 1.7190 - accuracy: 0.6957
Epoch 129/500
1/1 - 0s - loss: 1.6904 - accuracy: 0.6957
Epoch 130/500
1/1 - 0s - loss: 1.6618 - accuracy: 0.6957
Epoch 131/500
1/1 - 0s - loss: 1.6333 - accuracy: 0.7391
Epoch 132/500
1/1 - 0s - loss: 1.6047 - accuracy: 0.7391

Epoch 133/500
1/1 - 0s - loss: 1.5763 - accuracy: 0.7391
Epoch 134/500
1/1 - 0s - loss: 1.5480 - accuracy: 0.7391
Epoch 135/500
1/1 - 0s - loss: 1.5197 - accuracy: 0.7391
Epoch 136/500
1/1 - 0s - loss: 1.4916 - accuracy: 0.7826
Epoch 137/500
1/1 - 0s - loss: 1.4637 - accuracy: 0.8261
Epoch 138/500
1/1 - 0s - loss: 1.4359 - accuracy: 0.8261
Epoch 139/500
1/1 - 0s - loss: 1.4083 - accuracy: 0.8261
Epoch 140/500
1/1 - 0s - loss: 1.3808 - accuracy: 0.8261
Epoch 141/500
1/1 - 0s - loss: 1.3536 - accuracy: 0.8261
Epoch 142/500
1/1 - 0s - loss: 1.3266 - accuracy: 0.8696
Epoch 143/500
1/1 - 0s - loss: 1.2998 - accuracy: 0.9130
Epoch 144/500
1/1 - 0s - loss: 1.2733 - accuracy: 0.9130
Epoch 145/500
1/1 - 0s - loss: 1.2470 - accuracy: 0.9130
Epoch 146/500
1/1 - 0s - loss: 1.2211 - accuracy: 0.9130
Epoch 147/500
1/1 - 0s - loss: 1.1954 - accuracy: 0.9130
Epoch 148/500
1/1 - 0s - loss: 1.1700 - accuracy: 0.9130
Epoch 149/500
1/1 - 0s - loss: 1.1449 - accuracy: 0.9130
Epoch 150/500
1/1 - 0s - loss: 1.1202 - accuracy: 0.9130
Epoch 151/500
1/1 - 0s - loss: 1.0958 - accuracy: 0.9130
Epoch 152/500
1/1 - 0s - loss: 1.0717 - accuracy: 0.9130
Epoch 153/500
1/1 - 0s - loss: 1.0479 - accuracy: 0.9130
Epoch 154/500
1/1 - 0s - loss: 1.0246 - accuracy: 0.9130
Epoch 155/500
1/1 - 0s - loss: 1.0015 - accuracy: 0.9130
Epoch 156/500
1/1 - 0s - loss: 0.9789 - accuracy: 0.9130
Epoch 157/500
1/1 - 0s - loss: 0.9566 - accuracy: 0.9130
Epoch 158/500
1/1 - 0s - loss: 0.9346 - accuracy: 0.9130
Epoch 159/500
1/1 - 0s - loss: 0.9131 - accuracy: 0.9130
Epoch 160/500
1/1 - 0s - loss: 0.8919 - accuracy: 0.9130
Epoch 161/500
1/1 - 0s - loss: 0.8711 - accuracy: 0.9565
Epoch 162/500
1/1 - 0s - loss: 0.8507 - accuracy: 0.9565
Epoch 163/500
1/1 - 0s - loss: 0.8307 - accuracy: 0.9565
Epoch 164/500
1/1 - 0s - loss: 0.8110 - accuracy: 0.9565
Epoch 165/500
1/1 - 0s - loss: 0.7918 - accuracy: 0.9565
Epoch 166/500
1/1 - 0s - loss: 0.7729 - accuracy: 0.9565
Epoch 167/500
1/1 - 0s - loss: 0.7544 - accuracy: 0.9565
Epoch 168/500
1/1 - 0s - loss: 0.7363 - accuracy: 0.9565
Epoch 169/500
1/1 - 0s - loss: 0.7186 - accuracy: 0.9565
Epoch 170/500
1/1 - 0s - loss: 0.7013 - accuracy: 0.9565
Epoch 171/500
1/1 - 0s - loss: 0.6844 - accuracy: 0.9565
Epoch 172/500
1/1 - 0s - loss: 0.6678 - accuracy: 0.9565
Epoch 173/500
1/1 - 0s - loss: 0.6516 - accuracy: 0.9565
Epoch 174/500
1/1 - 0s - loss: 0.6358 - accuracy: 0.9565
Epoch 175/500
1/1 - 0s - loss: 0.6204 - accuracy: 0.9565
Epoch 176/500
1/1 - 0s - loss: 0.6053 - accuracy: 0.9565
Epoch 177/500
1/1 - 0s - loss: 0.5906 - accuracy: 0.9565
Epoch 178/500
1/1 - 0s - loss: 0.5762 - accuracy: 0.9565
Epoch 179/500
1/1 - 0s - loss: 0.5622 - accuracy: 0.9565
Epoch 180/500
1/1 - 0s - loss: 0.5486 - accuracy: 0.9565
Epoch 181/500
1/1 - 0s - loss: 0.5353 - accuracy: 0.9565
Epoch 182/500
1/1 - 0s - loss: 0.5224 - accuracy: 0.9565
Epoch 183/500
1/1 - 0s - loss: 0.5098 - accuracy: 0.9565
Epoch 184/500
1/1 - 0s - loss: 0.4975 - accuracy: 0.9565
Epoch 185/500
1/1 - 0s - loss: 0.4856 - accuracy: 0.9565
Epoch 186/500
1/1 - 0s - loss: 0.4740 - accuracy: 0.9565
Epoch 187/500
1/1 - 0s - loss: 0.4627 - accuracy: 0.9565
Epoch 188/500
1/1 - 0s - loss: 0.4517 - accuracy: 0.9565
Epoch 189/500
1/1 - 0s - loss: 0.4410 - accuracy: 0.9565
Epoch 190/500
1/1 - 0s - loss: 0.4306 - accuracy: 0.9565
Epoch 191/500
1/1 - 0s - loss: 0.4206 - accuracy: 0.9565
Epoch 192/500
1/1 - 0s - loss: 0.4108 - accuracy: 0.9565
Epoch 193/500
1/1 - 0s - loss: 0.4013 - accuracy: 0.9565
Epoch 194/500
1/1 - 0s - loss: 0.3921 - accuracy: 0.9565
Epoch 195/500
1/1 - 0s - loss: 0.3831 - accuracy: 0.9565
Epoch 196/500
1/1 - 0s - loss: 0.3744 - accuracy: 0.9565
Epoch 197/500
1/1 - 0s - loss: 0.3660 - accuracy: 0.9565
Epoch 198/500
1/1 - 0s - loss: 0.3578 - accuracy: 0.9565
Epoch 199/500
1/1 - 0s - loss: 0.3499 - accuracy: 0.9565
Epoch 200/500
1/1 - 0s - loss: 0.3422 - accuracy: 0.9565
Epoch 201/500
1/1 - 0s - loss: 0.3348 - accuracy: 0.9565
Epoch 202/500
1/1 - 0s - loss: 0.3275 - accuracy: 0.9565
Epoch 203/500
1/1 - 0s - loss: 0.3205 - accuracy: 0.9565
Epoch 204/500
1/1 - 0s - loss: 0.3137 - accuracy: 0.9565
Epoch 205/500
1/1 - 0s - loss: 0.3072 - accuracy: 0.9565
Epoch 206/500
1/1 - 0s - loss: 0.3008 - accuracy: 0.9565
Epoch 207/500
1/1 - 0s - loss: 0.2946 - accuracy: 0.9565
Epoch 208/500
1/1 - 0s - loss: 0.2886 - accuracy: 0.9565
Epoch 209/500
1/1 - 0s - loss: 0.2828 - accuracy: 0.9565
Epoch 210/500
1/1 - 0s - loss: 0.2771 - accuracy: 0.9565
Epoch 211/500
1/1 - 0s - loss: 0.2717 - accuracy: 0.9565
Epoch 212/500
1/1 - 0s - loss: 0.2664 - accuracy: 0.9565
Epoch 213/500
1/1 - 0s - loss: 0.2612 - accuracy: 0.9565
Epoch 214/500
1/1 - 0s - loss: 0.2562 - accuracy: 0.9565
Epoch 215/500
1/1 - 0s - loss: 0.2514 - accuracy: 0.9565
Epoch 216/500
1/1 - 0s - loss: 0.2467 - accuracy: 0.9565
Epoch 217/500
1/1 - 0s - loss: 0.2422 - accuracy: 0.9565
Epoch 218/500
1/1 - 0s - loss: 0.2378 - accuracy: 0.9565
Epoch 219/500
1/1 - 0s - loss: 0.2335 - accuracy: 0.9565
Epoch 220/500
1/1 - 0s - loss: 0.2294 - accuracy: 0.9565
Epoch 221/500
1/1 - 0s - loss: 0.2253 - accuracy: 0.9565
Epoch 222/500
1/1 - 0s - loss: 0.2214 - accuracy: 0.9565
Epoch 223/500
1/1 - 0s - loss: 0.2177 - accuracy: 0.9565
Epoch 224/500
1/1 - 0s - loss: 0.2140 - accuracy: 0.9565
Epoch 225/500
1/1 - 0s - loss: 0.2104 - accuracy: 0.9565
Epoch 226/500
1/1 - 0s - loss: 0.2070 - accuracy: 0.9565
Epoch 227/500
1/1 - 0s - loss: 0.2036 - accuracy: 0.9565
Epoch 228/500
1/1 - 0s - loss: 0.2004 - accuracy: 0.9565
Epoch 229/500
1/1 - 0s - loss: 0.1972 - accuracy: 0.9565
Epoch 230/500
1/1 - 0s - loss: 0.1941 - accuracy: 0.9565
Epoch 231/500
1/1 - 0s - loss: 0.1912 - accuracy: 0.9565
Epoch 232/500
1/1 - 0s - loss: 0.1883 - accuracy: 0.9565
Epoch 233/500
1/1 - 0s - loss: 0.1855 - accuracy: 0.9565
Epoch 234/500
1/1 - 0s - loss: 0.1828 - accuracy: 0.9565
Epoch 235/500
1/1 - 0s - loss: 0.1801 - accuracy: 0.9565
Epoch 236/500
1/1 - 0s - loss: 0.1776 - accuracy: 0.9565
Epoch 237/500
1/1 - 0s - loss: 0.1751 - accuracy: 0.9565
Epoch 238/500
1/1 - 0s - loss: 0.1727 - accuracy: 0.9565
Epoch 239/500
1/1 - 0s - loss: 0.1703 - accuracy: 0.9565
Epoch 240/500
1/1 - 0s - loss: 0.1680 - accuracy: 0.9565
Epoch 241/500
1/1 - 0s - loss: 0.1658 - accuracy: 0.9565
Epoch 242/500
1/1 - 0s - loss: 0.1637 - accuracy: 0.9565
Epoch 243/500
1/1 - 0s - loss: 0.1616 - accuracy: 0.9565
Epoch 244/500
1/1 - 0s - loss: 0.1596 - accuracy: 0.9565
Epoch 245/500
1/1 - 0s - loss: 0.1576 - accuracy: 0.9565
Epoch 246/500
1/1 - 0s - loss: 0.1557 - accuracy: 0.9565
Epoch 247/500
1/1 - 0s - loss: 0.1538 - accuracy: 0.9565
Epoch 248/500
1/1 - 0s - loss: 0.1520 - accuracy: 0.9565
Epoch 249/500
1/1 - 0s - loss: 0.1503 - accuracy: 0.9565
Epoch 250/500
1/1 - 0s - loss: 0.1486 - accuracy: 0.9565
Epoch 251/500
1/1 - 0s - loss: 0.1469 - accuracy: 0.9565
Epoch 252/500
1/1 - 0s - loss: 0.1453 - accuracy: 0.9565
Epoch 253/500
1/1 - 0s - loss: 0.1437 - accuracy: 0.9565
Epoch 254/500
1/1 - 0s - loss: 0.1422 - accuracy: 0.9565
Epoch 255/500
1/1 - 0s - loss: 0.1407 - accuracy: 0.9565
Epoch 256/500
1/1 - 0s - loss: 0.1393 - accuracy: 0.9565
Epoch 257/500
1/1 - 0s - loss: 0.1379 - accuracy: 0.9565
Epoch 258/500
1/1 - 0s - loss: 0.1365 - accuracy: 0.9565
Epoch 259/500
1/1 - 0s - loss: 0.1352 - accuracy: 0.9565
Epoch 260/500
1/1 - 0s - loss: 0.1339 - accuracy: 0.9565
Epoch 261/500
1/1 - 0s - loss: 0.1326 - accuracy: 0.9565
Epoch 262/500
1/1 - 0s - loss: 0.1314 - accuracy: 0.9565
Epoch 263/500
1/1 - 0s - loss: 0.1302 - accuracy: 0.9565
Epoch 264/500
1/1 - 0s - loss: 0.1290 - accuracy: 0.9565
Epoch 265/500
1/1 - 0s - loss: 0.1279 - accuracy: 0.9565
Epoch 266/500
1/1 - 0s - loss: 0.1268 - accuracy: 0.9565
Epoch 267/500
1/1 - 0s - loss: 0.1257 - accuracy: 0.9565
Epoch 268/500
1/1 - 0s - loss: 0.1246 - accuracy: 0.9565
Epoch 269/500
1/1 - 0s - loss: 0.1236 - accuracy: 0.9565
Epoch 270/500
1/1 - 0s - loss: 0.1226 - accuracy: 0.9565
Epoch 271/500
1/1 - 0s - loss: 0.1217 - accuracy: 0.9565
Epoch 272/500
1/1 - 0s - loss: 0.1207 - accuracy: 0.9565
Epoch 273/500
1/1 - 0s - loss: 0.1198 - accuracy: 0.9565
Epoch 274/500
1/1 - 0s - loss: 0.1189 - accuracy: 0.9565
Epoch 275/500
1/1 - 0s - loss: 0.1180 - accuracy: 0.9565
Epoch 276/500
1/1 - 0s - loss: 0.1171 - accuracy: 0.9565

Epoch 277/500
1/1 - 0s - loss: 0.1163 - accuracy: 0.9565
Epoch 278/500
1/1 - 0s - loss: 0.1155 - accuracy: 0.9565
Epoch 279/500
1/1 - 0s - loss: 0.1147 - accuracy: 0.9565
Epoch 280/500
1/1 - 0s - loss: 0.1139 - accuracy: 0.9565
Epoch 281/500
1/1 - 0s - loss: 0.1131 - accuracy: 0.9565
Epoch 282/500
1/1 - 0s - loss: 0.1124 - accuracy: 0.9565
Epoch 283/500
1/1 - 0s - loss: 0.1117 - accuracy: 0.9565
Epoch 284/500
1/1 - 0s - loss: 0.1109 - accuracy: 0.9565
Epoch 285/500
1/1 - 0s - loss: 0.1102 - accuracy: 0.9565
Epoch 286/500
1/1 - 0s - loss: 0.1096 - accuracy: 0.9565
Epoch 287/500
1/1 - 0s - loss: 0.1089 - accuracy: 0.9565
Epoch 288/500
1/1 - 0s - loss: 0.1082 - accuracy: 0.9565
Epoch 289/500
1/1 - 0s - loss: 0.1076 - accuracy: 0.9565
Epoch 290/500
1/1 - 0s - loss: 0.1070 - accuracy: 0.9565
Epoch 291/500
1/1 - 0s - loss: 0.1064 - accuracy: 0.9565
Epoch 292/500
1/1 - 0s - loss: 0.1058 - accuracy: 0.9565
Epoch 293/500
1/1 - 0s - loss: 0.1052 - accuracy: 0.9565
Epoch 294/500
1/1 - 0s - loss: 0.1046 - accuracy: 0.9565
Epoch 295/500
1/1 - 0s - loss: 0.1041 - accuracy: 0.9565
Epoch 296/500
1/1 - 0s - loss: 0.1035 - accuracy: 0.9565
Epoch 297/500
1/1 - 0s - loss: 0.1030 - accuracy: 0.9565
Epoch 298/500
1/1 - 0s - loss: 0.1024 - accuracy: 0.9565
Epoch 299/500
1/1 - 0s - loss: 0.1019 - accuracy: 0.9565
Epoch 300/500
1/1 - 0s - loss: 0.1014 - accuracy: 0.9565
Epoch 301/500
1/1 - 0s - loss: 0.1009 - accuracy: 0.9565
Epoch 302/500
1/1 - 0s - loss: 0.1004 - accuracy: 0.9565
Epoch 303/500
1/1 - 0s - loss: 0.1000 - accuracy: 0.9565
Epoch 304/500
1/1 - 0s - loss: 0.0995 - accuracy: 0.9565
Epoch 305/500
1/1 - 0s - loss: 0.0990 - accuracy: 0.9565
Epoch 306/500
1/1 - 0s - loss: 0.0986 - accuracy: 0.9565
Epoch 307/500
1/1 - 0s - loss: 0.0981 - accuracy: 0.9565
Epoch 308/500
1/1 - 0s - loss: 0.0977 - accuracy: 0.9565
Epoch 309/500
1/1 - 0s - loss: 0.0973 - accuracy: 0.9565
Epoch 310/500
1/1 - 0s - loss: 0.0969 - accuracy: 0.9565
Epoch 311/500
1/1 - 0s - loss: 0.0965 - accuracy: 0.9565
Epoch 312/500
1/1 - 0s - loss: 0.0961 - accuracy: 0.9565
Epoch 313/500
1/1 - 0s - loss: 0.0957 - accuracy: 0.9565
Epoch 314/500
1/1 - 0s - loss: 0.0953 - accuracy: 0.9565
Epoch 315/500
1/1 - 0s - loss: 0.0949 - accuracy: 0.9565
Epoch 316/500
1/1 - 0s - loss: 0.0945 - accuracy: 0.9565
Epoch 317/500
1/1 - 0s - loss: 0.0942 - accuracy: 0.9565
Epoch 318/500
1/1 - 0s - loss: 0.0938 - accuracy: 0.9565
Epoch 319/500
1/1 - 0s - loss: 0.0934 - accuracy: 0.9565
Epoch 320/500
1/1 - 0s - loss: 0.0931 - accuracy: 0.9565
Epoch 321/500
1/1 - 0s - loss: 0.0927 - accuracy: 0.9565
Epoch 322/500
1/1 - 0s - loss: 0.0924 - accuracy: 0.9565
Epoch 323/500
1/1 - 0s - loss: 0.0921 - accuracy: 0.9565
Epoch 324/500
1/1 - 0s - loss: 0.0918 - accuracy: 0.9565
Epoch 325/500
1/1 - 0s - loss: 0.0914 - accuracy: 0.9565
Epoch 326/500
1/1 - 0s - loss: 0.0911 - accuracy: 0.9565
Epoch 327/500
1/1 - 0s - loss: 0.0908 - accuracy: 0.9565
Epoch 328/500
1/1 - 0s - loss: 0.0905 - accuracy: 0.9565
Epoch 329/500
1/1 - 0s - loss: 0.0902 - accuracy: 0.9565
Epoch 330/500
1/1 - 0s - loss: 0.0899 - accuracy: 0.9565
Epoch 331/500
1/1 - 0s - loss: 0.0896 - accuracy: 0.9565
Epoch 332/500
1/1 - 0s - loss: 0.0893 - accuracy: 0.9565
Epoch 333/500
1/1 - 0s - loss: 0.0891 - accuracy: 0.9565
Epoch 334/500
1/1 - 0s - loss: 0.0888 - accuracy: 0.9565
Epoch 335/500
1/1 - 0s - loss: 0.0885 - accuracy: 0.9565
Epoch 336/500
1/1 - 0s - loss: 0.0882 - accuracy: 0.9565
Epoch 337/500
1/1 - 0s - loss: 0.0880 - accuracy: 0.9565
Epoch 338/500
1/1 - 0s - loss: 0.0877 - accuracy: 0.9565
Epoch 339/500
1/1 - 0s - loss: 0.0875 - accuracy: 0.9565
Epoch 340/500
1/1 - 0s - loss: 0.0872 - accuracy: 0.9565
Epoch 341/500
1/1 - 0s - loss: 0.0870 - accuracy: 0.9565
Epoch 342/500
1/1 - 0s - loss: 0.0867 - accuracy: 0.9565
Epoch 343/500
1/1 - 0s - loss: 0.0865 - accuracy: 0.9565
Epoch 344/500
1/1 - 0s - loss: 0.0862 - accuracy: 0.9565
Epoch 345/500
1/1 - 0s - loss: 0.0860 - accuracy: 0.9565
Epoch 346/500
1/1 - 0s - loss: 0.0858 - accuracy: 0.9565
Epoch 347/500
1/1 - 0s - loss: 0.0856 - accuracy: 0.9565
Epoch 348/500
1/1 - 0s - loss: 0.0853 - accuracy: 0.9565
Epoch 349/500
1/1 - 0s - loss: 0.0851 - accuracy: 0.9565
Epoch 350/500
1/1 - 0s - loss: 0.0849 - accuracy: 0.9565
Epoch 351/500
1/1 - 0s - loss: 0.0847 - accuracy: 0.9565
Epoch 352/500
1/1 - 0s - loss: 0.0845 - accuracy: 0.9565
Epoch 353/500
1/1 - 0s - loss: 0.0843 - accuracy: 0.9565
Epoch 354/500
1/1 - 0s - loss: 0.0841 - accuracy: 0.9565
Epoch 355/500
1/1 - 0s - loss: 0.0839 - accuracy: 0.9565
Epoch 356/500
1/1 - 0s - loss: 0.0837 - accuracy: 0.9565
Epoch 357/500
1/1 - 0s - loss: 0.0835 - accuracy: 0.9565
Epoch 358/500
1/1 - 0s - loss: 0.0833 - accuracy: 0.9565
Epoch 359/500
1/1 - 0s - loss: 0.0831 - accuracy: 0.9565
Epoch 360/500
1/1 - 0s - loss: 0.0829 - accuracy: 0.9565
Epoch 361/500
1/1 - 0s - loss: 0.0827 - accuracy: 0.9565
Epoch 362/500
1/1 - 0s - loss: 0.0825 - accuracy: 0.9565
Epoch 363/500
1/1 - 0s - loss: 0.0823 - accuracy: 0.9565
Epoch 364/500
1/1 - 0s - loss: 0.0821 - accuracy: 0.9565
Epoch 365/500
1/1 - 0s - loss: 0.0820 - accuracy: 0.9565
Epoch 366/500
1/1 - 0s - loss: 0.0818 - accuracy: 0.9565
Epoch 367/500
1/1 - 0s - loss: 0.0816 - accuracy: 0.9565
Epoch 368/500
1/1 - 0s - loss: 0.0815 - accuracy: 0.9565
Epoch 369/500
1/1 - 0s - loss: 0.0813 - accuracy: 0.9565
Epoch 370/500
1/1 - 0s - loss: 0.0811 - accuracy: 0.9565
Epoch 371/500
1/1 - 0s - loss: 0.0810 - accuracy: 0.9565
Epoch 372/500
1/1 - 0s - loss: 0.0808 - accuracy: 0.9565
Epoch 373/500
1/1 - 0s - loss: 0.0806 - accuracy: 0.9565
Epoch 374/500
1/1 - 0s - loss: 0.0805 - accuracy: 0.9565
Epoch 375/500
1/1 - 0s - loss: 0.0803 - accuracy: 0.9565
Epoch 376/500
1/1 - 0s - loss: 0.0802 - accuracy: 0.9565
Epoch 377/500
1/1 - 0s - loss: 0.0800 - accuracy: 0.9565
Epoch 378/500
1/1 - 0s - loss: 0.0799 - accuracy: 0.9565
Epoch 379/500
1/1 - 0s - loss: 0.0797 - accuracy: 0.9565
Epoch 380/500
1/1 - 0s - loss: 0.0796 - accuracy: 0.9565
Epoch 381/500
1/1 - 0s - loss: 0.0794 - accuracy: 0.9565
Epoch 382/500
1/1 - 0s - loss: 0.0793 - accuracy: 0.9565
Epoch 383/500
1/1 - 0s - loss: 0.0792 - accuracy: 0.9565
Epoch 384/500
1/1 - 0s - loss: 0.0790 - accuracy: 0.9565
Epoch 385/500
1/1 - 0s - loss: 0.0789 - accuracy: 0.9565
Epoch 386/500
1/1 - 0s - loss: 0.0787 - accuracy: 0.9565
Epoch 387/500
1/1 - 0s - loss: 0.0786 - accuracy: 0.9565
Epoch 388/500
1/1 - 0s - loss: 0.0785 - accuracy: 0.9565
Epoch 389/500
1/1 - 0s - loss: 0.0783 - accuracy: 0.9565
Epoch 390/500
1/1 - 0s - loss: 0.0782 - accuracy: 0.9565
Epoch 391/500
1/1 - 0s - loss: 0.0781 - accuracy: 0.9565
Epoch 392/500
1/1 - 0s - loss: 0.0780 - accuracy: 0.9565
Epoch 393/500
1/1 - 0s - loss: 0.0778 - accuracy: 0.9565
Epoch 394/500
1/1 - 0s - loss: 0.0777 - accuracy: 0.9565
Epoch 395/500
1/1 - 0s - loss: 0.0776 - accuracy: 0.9565
Epoch 396/500
1/1 - 0s - loss: 0.0775 - accuracy: 0.9565
Epoch 397/500
1/1 - 0s - loss: 0.0773 - accuracy: 0.9565
Epoch 398/500
1/1 - 0s - loss: 0.0772 - accuracy: 0.9565
Epoch 399/500
1/1 - 0s - loss: 0.0771 - accuracy: 0.9565
Epoch 400/500
1/1 - 0s - loss: 0.0770 - accuracy: 0.9565
Epoch 401/500
1/1 - 0s - loss: 0.0769 - accuracy: 0.9565
Epoch 402/500
1/1 - 0s - loss: 0.0768 - accuracy: 0.9565
Epoch 403/500
1/1 - 0s - loss: 0.0767 - accuracy: 0.9565
Epoch 404/500
1/1 - 0s - loss: 0.0765 - accuracy: 0.9565
Epoch 405/500
1/1 - 0s - loss: 0.0764 - accuracy: 0.9565
Epoch 406/500
1/1 - 0s - loss: 0.0763 - accuracy: 0.9565
Epoch 407/500
1/1 - 0s - loss: 0.0762 - accuracy: 0.9565
Epoch 408/500
1/1 - 0s - loss: 0.0761 - accuracy: 0.9565
Epoch 409/500
1/1 - 0s - loss: 0.0760 - accuracy: 0.9565
Epoch 410/500
1/1 - 0s - loss: 0.0759 - accuracy: 0.9565
Epoch 411/500
1/1 - 0s - loss: 0.0758 - accuracy: 0.9565
Epoch 412/500
1/1 - 0s - loss: 0.0757 - accuracy: 0.9565
Epoch 413/500
1/1 - 0s - loss: 0.0756 - accuracy: 0.9565
Epoch 414/500
1/1 - 0s - loss: 0.0755 - accuracy: 0.9565
Epoch 415/500
1/1 - 0s - loss: 0.0754 - accuracy: 0.9565
Epoch 416/500
1/1 - 0s - loss: 0.0753 - accuracy: 0.9565
Epoch 417/500
1/1 - 0s - loss: 0.0752 - accuracy: 0.9565
Epoch 418/500
1/1 - 0s - loss: 0.0751 - accuracy: 0.9565
Epoch 419/500
1/1 - 0s - loss: 0.0750 - accuracy: 0.9565
Epoch 420/500
1/1 - 0s - loss: 0.0749 - accuracy: 0.9565

Epoch 421/500
1/1 - 0s - loss: 0.0748 - accuracy: 0.9565
Epoch 422/500
1/1 - 0s - loss: 0.0747 - accuracy: 0.9565
Epoch 423/500
1/1 - 0s - loss: 0.0746 - accuracy: 0.9565
Epoch 424/500
1/1 - 0s - loss: 0.0746 - accuracy: 0.9565
Epoch 425/500
1/1 - 0s - loss: 0.0745 - accuracy: 0.9565
Epoch 426/500
1/1 - 0s - loss: 0.0744 - accuracy: 0.9565
Epoch 427/500
1/1 - 0s - loss: 0.0743 - accuracy: 0.9565
Epoch 428/500
1/1 - 0s - loss: 0.0742 - accuracy: 0.9565
Epoch 429/500
1/1 - 0s - loss: 0.0741 - accuracy: 0.9565
Epoch 430/500
1/1 - 0s - loss: 0.0740 - accuracy: 0.9565
Epoch 431/500
1/1 - 0s - loss: 0.0740 - accuracy: 0.9565
Epoch 432/500
1/1 - 0s - loss: 0.0739 - accuracy: 0.9565
Epoch 433/500
1/1 - 0s - loss: 0.0738 - accuracy: 0.9565
Epoch 434/500
1/1 - 0s - loss: 0.0737 - accuracy: 0.9565
Epoch 435/500
1/1 - 0s - loss: 0.0736 - accuracy: 0.9565
Epoch 436/500
1/1 - 0s - loss: 0.0735 - accuracy: 0.9565
Epoch 437/500
1/1 - 0s - loss: 0.0735 - accuracy: 0.9565
Epoch 438/500
1/1 - 0s - loss: 0.0734 - accuracy: 0.9565
Epoch 439/500
1/1 - 0s - loss: 0.0733 - accuracy: 0.9565
Epoch 440/500
1/1 - 0s - loss: 0.0732 - accuracy: 0.9565
Epoch 441/500
1/1 - 0s - loss: 0.0732 - accuracy: 0.9565
Epoch 442/500
1/1 - 0s - loss: 0.0731 - accuracy: 0.9565
Epoch 443/500
1/1 - 0s - loss: 0.0730 - accuracy: 0.9565
Epoch 444/500
1/1 - 0s - loss: 0.0729 - accuracy: 0.9565
Epoch 445/500
1/1 - 0s - loss: 0.0729 - accuracy: 0.9565
Epoch 446/500
1/1 - 0s - loss: 0.0728 - accuracy: 0.9565
Epoch 447/500
1/1 - 0s - loss: 0.0727 - accuracy: 0.9565
Epoch 448/500
1/1 - 0s - loss: 0.0726 - accuracy: 0.9565
Epoch 449/500
1/1 - 0s - loss: 0.0726 - accuracy: 0.9565
Epoch 450/500
1/1 - 0s - loss: 0.0725 - accuracy: 0.9565
Epoch 451/500
1/1 - 0s - loss: 0.0724 - accuracy: 0.9565
Epoch 452/500
1/1 - 0s - loss: 0.0724 - accuracy: 0.9565
Epoch 453/500
1/1 - 0s - loss: 0.0723 - accuracy: 0.9565
Epoch 454/500
1/1 - 0s - loss: 0.0722 - accuracy: 0.9565
Epoch 455/500
1/1 - 0s - loss: 0.0722 - accuracy: 0.9565
Epoch 456/500
1/1 - 0s - loss: 0.0721 - accuracy: 0.9565
Epoch 457/500
1/1 - 0s - loss: 0.0720 - accuracy: 0.9565
Epoch 458/500
1/1 - 0s - loss: 0.0720 - accuracy: 0.9565
Epoch 459/500
1/1 - 0s - loss: 0.0719 - accuracy: 0.9565
Epoch 460/500
1/1 - 0s - loss: 0.0718 - accuracy: 0.9565
Epoch 461/500
1/1 - 0s - loss: 0.0718 - accuracy: 0.9565
Epoch 462/500
1/1 - 0s - loss: 0.0717 - accuracy: 0.9565
Epoch 463/500
1/1 - 0s - loss: 0.0717 - accuracy: 0.9565
Epoch 464/500
1/1 - 0s - loss: 0.0716 - accuracy: 0.9565
Epoch 465/500
1/1 - 0s - loss: 0.0715 - accuracy: 0.9565
Epoch 466/500
1/1 - 0s - loss: 0.0715 - accuracy: 0.9565
Epoch 467/500
1/1 - 0s - loss: 0.0714 - accuracy: 0.9565
Epoch 468/500
1/1 - 0s - loss: 0.0713 - accuracy: 0.9565
Epoch 469/500
1/1 - 0s - loss: 0.0713 - accuracy: 0.9565
Epoch 470/500
1/1 - 0s - loss: 0.0712 - accuracy: 0.9565
Epoch 471/500
1/1 - 0s - loss: 0.0712 - accuracy: 0.9565
Epoch 472/500
1/1 - 0s - loss: 0.0711 - accuracy: 0.9565
Epoch 473/500
1/1 - 0s - loss: 0.0711 - accuracy: 0.9565
Epoch 474/500
1/1 - 0s - loss: 0.0710 - accuracy: 0.9565
Epoch 475/500
1/1 - 0s - loss: 0.0709 - accuracy: 0.9565
Epoch 476/500
1/1 - 0s - loss: 0.0709 - accuracy: 0.9565
Epoch 477/500
1/1 - 0s - loss: 0.0708 - accuracy: 0.9565
Epoch 478/500
1/1 - 0s - loss: 0.0708 - accuracy: 0.9565
Epoch 479/500
1/1 - 0s - loss: 0.0707 - accuracy: 0.9565
Epoch 480/500
1/1 - 0s - loss: 0.0707 - accuracy: 0.9565
Epoch 481/500
1/1 - 0s - loss: 0.0706 - accuracy: 0.9565
Epoch 482/500
1/1 - 0s - loss: 0.0706 - accuracy: 0.9565
Epoch 483/500
1/1 - 0s - loss: 0.0705 - accuracy: 0.9565
Epoch 484/500
1/1 - 0s - loss: 0.0705 - accuracy: 0.9565
Epoch 485/500
1/1 - 0s - loss: 0.0704 - accuracy: 0.9565
Epoch 486/500
1/1 - 0s - loss: 0.0704 - accuracy: 0.9565
Epoch 487/500
1/1 - 0s - loss: 0.0703 - accuracy: 0.9565
Epoch 488/500
1/1 - 0s - loss: 0.0703 - accuracy: 0.9565
Epoch 489/500
1/1 - 0s - loss: 0.0702 - accuracy: 0.9565
Epoch 490/500
1/1 - 0s - loss: 0.0702 - accuracy: 0.9565
Epoch 491/500
1/1 - 0s - loss: 0.0701 - accuracy: 0.9565
Epoch 492/500
1/1 - 0s - loss: 0.0701 - accuracy: 0.9565
Epoch 493/500
1/1 - 0s - loss: 0.0700 - accuracy: 0.9565
Epoch 494/500
1/1 - 0s - loss: 0.0700 - accuracy: 0.9565
Epoch 495/500
1/1 - 0s - loss: 0.0699 - accuracy: 0.9565
Epoch 496/500
1/1 - 0s - loss: 0.0699 - accuracy: 0.9565
Epoch 497/500
1/1 - 0s - loss: 0.0698 - accuracy: 0.9565
Epoch 498/500
1/1 - 0s - loss: 0.0698 - accuracy: 0.9565
Epoch 499/500
1/1 - 0s - loss: 0.0697 - accuracy: 0.9565
Epoch 500/500
1/1 - 0s - loss: 0.0697 - accuracy: 0.9565

<tensorflow.python.keras.callbacks.History at 0x7fed32759e80>

plot_model(model)

Failed to import pydot. You must install pydot and graphviz for `pydotprint` to work.

# evaluate model
print(generate_seq(model, tokenizer, max_length-1, 'Jack and', 5))
print(generate_seq(model, tokenizer, max_length-1, 'And Jill', 3))
print(generate_seq(model, tokenizer, max_length-1, 'fell down', 5))
print(generate_seq(model, tokenizer, max_length-1, 'pail of', 5))

Jack and jill went up the hill
And Jill went up the
fell down and broke his crown and
pail of water jack fell down and