restructured

This commit is contained in:
Nick Walton
2019-11-18 16:40:45 -07:00
parent fb2ccbf7cf
commit ca7e5fe6bc
101 changed files with 5 additions and 90822 deletions
-14
View File
@@ -1,14 +0,0 @@
FROM nvcr.io/nvidia/tensorflow:19.08-py3
ENV APP_HOME /app
WORKDIR $APP_HOME
COPY . ./
RUN pip install -r requirements.txt
WORKDIR generator/ctrl
RUN ls
RUN sh install_ctrl_py3.sh
RUN sh download_model.sh
WORKDIR $APP_HOME
CMD exec guincorn --bind :%PORT --workers 1 --threads 8 app:app
@@ -1,7 +1,6 @@
from story.story_manager import *
# from generator.web.web_generator import *
# from generator.ctrl.ctrl_generator import *
from generator.gpt2.gpt2_generator import *
CRED_FILE = "./AI-Adventure-2bb65e3a4e2f.json"
@@ -2,10 +2,7 @@ import tensorflow as tf
import numpy as np
tf.enable_eager_execution()
import generator.ctrl.model.transformer as transformer
import re
from collections import Counter
from tensorflow.python import debug as tf_debug
import aidungeon.generator.ctrl.model.transformer as transformer
from tensorflow.python.ops import math_ops
from tensorflow.python.ops import embedding_ops
import fastBPE
@@ -2,12 +2,8 @@ from story.utils import *
import warnings
warnings.filterwarnings("ignore")
import os
import requests
import sys
import tensorflow as tf
import requests
from tqdm import tqdm
from generator.gpt2.src import model, sample, encoder
from aidungeon.generator.gpt2.src import sample, encoder, model
import json
import numpy as np
@@ -1,6 +1,7 @@
import tensorflow as tf
from generator.gpt2.src import model
from aidungeon.generator.gpt2.src import model
def penalize_used(logits, output):
View File
@@ -1,4 +1,3 @@
from generator.tf.src.encoder import *
import googleapiclient.discovery
import traceback
View File
View File
-19
View File
@@ -1,19 +0,0 @@
runtime: python37
entrypoint: gunicorn --timeout=300 --graceful-timeout=300 -b :$PORT main:app
instance_class: F1
automatic_scaling:
min_instances: 1
handlers:
# This configures Google App Engine to serve the files in the app's static
# directory.
- url: /static
static_dir: static
# This handler routes all requests not caught above to your main app. It is
# required when static routes are defined, but can be omitted (along with
# the entire handlers section) when there are no static files defined.
- url: /.*
script: auto
-4
View File
@@ -1,4 +0,0 @@
steps:
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy", "--no-promote"]
timeout: "1600s"
+1 -1
View File
@@ -1 +1 @@
gsutil -m cp -r gs://aidungeon2model/model_v1 ./generator/gpt2/models
gsutil -m cp -r gs://aidungeon2model/model_v1 ./aidungeon/generator/gpt2/models
@@ -1,7 +0,0 @@
{
"n_vocab": 50257,
"n_ctx": 1024,
"n_embd": 1600,
"n_head": 25,
"n_layer": 48
}
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large Load Diff
-1
View File
@@ -1 +0,0 @@
gsutil -m cp -r gs://aidungeon2model/model_v1 ./generator/gpt2/models
-67
View File
@@ -1,67 +0,0 @@
from flask import g
from flask import session
import os
from story.utils import *
import json
from flask import Flask, render_template, request, abort
from story.story_manager import *
import numpy as np
from story.story_manager import *
from generator.gpt2.gpt2_generator import *
app = Flask(__name__)
app.secret_key = '#d\xe0\xd1\xfb\xee\xa2\xbb\xd2\xf1/e)\xb5g\xdd<`\xc7\xa6\xa0-\xb8d0S'
generator = GPT2Generator()
story_manager = UnconstrainedStoryManager(generator)
# Shows about. (Should also link to paper when published)
@app.route('/about.html')
def about():
return render_template('about.html')
# Bread and butter of app, updates story and returns based on choice
@app.route('/generate', methods=['POST'])
def generate():
action = request.form["action"]
# If there is no story in session, make a new one
if "story" not in session or session["story"] is None:
print("Starting new story")
prompt = get_story_start("knight")
context = get_context("knight")
story_manager.start_new_story(prompt, context=context)
response = "\n" + context + str(story_manager.story) + "\n> "
# If there is a story in session continue from it.
else:
print("Using existing story")
story = session["story"]
story_manager.load_story(story, from_json=True)
if action != "":
action = action.strip()
action = action[0].upper() + action[1:]
action = "\n> " + action + "\n"
#action = remove_profanity(action)
response = story_manager.act(action) + "\n> "
session["story"] = story_manager.json_story()
print("Returning response")
return response
# Routes to index
@app.route('/')
def root():
session["story"] = None
return render_template('index.html')
def run_app():
app.run(host='0.0.0.0', port=8092)
if __name__ == '__main__':
run_app()
-4
View File
@@ -1,4 +0,0 @@
google-cloud-storage
numpy
regex
profanityfilter
-40000
View File
File diff suppressed because it is too large Load Diff
-1
View File
@@ -1 +0,0 @@
./gunicorn --timeout=300 --graceful-timeout=300 -b 0.0.0.0:8010 main:app
-3
View File
@@ -1,3 +0,0 @@
<span id="a">Adventurer@DungeonDream</span>:<span id="b">~</span><span id="c">$</span>
Welcome to the Dungeon.<!-- laglaglaglag--><p>Here you will embark on an epic journey through the dreams of an AI to claim victory on your quest. </p><!-- qowifjqwoeijfoqweijfqweoifjqweofijqweoqwoijefoqwijefoijfqiwoefjj -->
<p>Good luck</p>
Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

-285
View File
@@ -1,285 +0,0 @@
start_text = "<span id='a'>Adventurer@AIDungeon</span>:<span id='b'>~</span><span id='c'>$</span> ./EnterDungeon \n <br/><!-- laglaglaglaglaglaglaglaglaglaglag-->"
var acceptInput=false
var action_waiting = false
var inputStr = ""
var typing = false
var action_list = ["You attack", "You tell", "You use", "You go"]
var prompt_num = 0
var seed_max = 100
var seed_min = 0;
prompts = ["You enter a dungeon with your trusty sword and shield. You are searching for the evil necromancer who killed your family. You've heard that he resides at the bottom of the dungeon, guarded by legions of the undead. You enter the first door and see"]
if(seed == -1){
var seed = Math.floor(Math.random() * (+seed_max - +seed_min)) + +seed_min;
}
//var seed = 999
console.log("Seed is ", seed)
function isMobileDevice() {
return /Android|webOS|iPhone|iPad|iPod|BlackBerry|IEMobile|Opera Mini/i.test(navigator.userAgent)
};
function buttonCheck(){
if(typing == true){
setTimeout(buttonCheck, 500);
}
else{
document.getElementById('buttons').style.visibility='visible';
}
}
var StoryTracker = {
lastActionResult: "",
actions: [],
results: [],
choices: [],
// Requests the first story
getFirstStory:function(){
console.log("Requesting first story")
Typer.appendToText(prompts[prompt_num])
StoryTracker.requestFirstStory()
},
addFirstStory:function(story){
StoryTracker.lastActionResult = story
StoryTracker.makeActionRequests()
Typer.appendToText(story)
},
// Called after requesting options, prints generating msg if waits too lng
actionWait:function(){
if(action_waiting == true){
if(typing == true || acceptInput == true){
setTimeout(StoryTracker.actionWait, 10000);
}
else{
Typer.appendToText("\n\n Generating options... (~20s)")
}
}
},
// Callback for action request
addNextAction:function(action_result){
// Response receieved no longer waiting
var action_results = JSON.parse(action_result)
Typer.appendToText("\n\nOptions:")
action_waiting = false
StoryTracker.actions = []
StoryTracker.results = []
for (i = 0; i < 4; i++){
action_result = action_results[i]
action = action_result[0]
result = action_result[1]
StoryTracker.results.push(result)
var print_action = "\n" + String(i) + ") " + action
Typer.appendToText(print_action)
if (i == 3){
Typer.appendToText("\nWhich action do you choose? ")
// Now we wait for the user to give input to us.
acceptInput = true
if(isMobileDevice()){
setTimeout(buttonCheck, 500);
}
}
}
},
// Make a request to the server for result actions
makeActionRequests:function(){
action_waiting = true
setTimeout(StoryTracker.actionWait, 10000);
StoryTracker.requestActions(StoryTracker.lastActionResult, JSON.stringify(StoryTracker.choices))
},
requestFirstStory:function(){
$.post("/generate", {actions: false, seed, prompt_num},
StoryTracker.addFirstStory)
},
requestActions:function(last_action_result, choices){
$.post("/generate", {actions: true, seed, prompt_num, last_action_result, choices},
StoryTracker.addNextAction)
},
// Called once a choice has been made by button or entering.
processInput:function(){
var choice_int = parseInt(inputStr, 10)
if(choice_int >= 0 && choice_int <= 3){
console.log("choice_int is %d", choice_int)
StoryTracker.choices.push(choice_int)
StoryTracker.lastActionResult = StoryTracker.results[choice_int]
StoryTracker.makeActionRequests(StoryTracker.firstStory + StoryTracker.lastStory)
action_waiting = true
Typer.appendToText("\n")
Typer.appendToText(StoryTracker.lastActionResult)
}
else{
Typer.appendToText("Invalid choice. Must be a number from 0 to 3. \n")
Typer.appendToText("\nWhich action do you choose? ")
acceptInput = true
if(isMobileDevice()){
setTimeout(buttonCheck, 500);
}
}
inputStr = ""
}
}
// Used to control the terminal like screen typing
var Typer={
text: null,
index:0,
speed:2,
content:function(){
return $("#console").html()
},
appendToText:function(str){
str = str.replace(".", "." + "<!-- laglaglag-->")
typing = true
Typer.text = Typer.text + str;
},
removeChar:function(){
var cont=Typer.content()
$("#console").html($("#console").html().substring(0,cont.length-1))
Typer.text = Typer.text.substring(0, Typer.text.length-1)
Typer.index = Typer.index - 1
},
addText:function(){
if (Typer.index <= Typer.text.length) {
var cont=Typer.content()
if(cont.substring(cont.length-1,cont.length)=="|")
$("#console").html($("#console").html().substring(0,cont.length-1))
if (Typer.text.substring(Typer.index, Typer.index + Typer.speed).includes(".")){
Typer.index += 1
}
else{
Typer.index+=Typer.speed
}
var text=Typer.text.substring(0,Typer.index)
var rtn= new RegExp("\n", "g")
$("#console").html(text.replace(rtn,"<br/>"))
}
else{
typing = false
}
},
}
function writeAppend(str){
$("#console").append(str)
}
function startTyping(){
addTextTimer = setInterval("typeWords()", 20)
}
function typeWords() {
Typer.addText()
}
document.addEventListener("keydown", KeyCheck);
function KeyCheck(evt) {
evt = evt || window.event
var charCode = evt.keyCode || evt.which
if(charCode == 8){
if(inputStr.length > 0){
console.log(inputStr)
inputStr = inputStr.substring(0, inputStr.length-1)
console.log(inputStr)
Typer.removeChar()
}
}
}
document.onkeypress = function(evt) {
if(acceptInput && !isMobileDevice()){
evt = evt || window.event
var charCode = evt.keyCode || evt.which
if(charCode == 13){
acceptInput = false
Typer.appendToText("\n")
StoryTracker.processInput(inputStr)
}
else{
var charStr = String.fromCharCode(charCode)
Typer.appendToText(charStr)
inputStr = inputStr + charStr
}
}
}
function onButtonClick(num){
document.getElementById('buttons').style.visibility='hidden';
if (acceptInput == true){
acceptInput = false
num = String(num)
Typer.appendToText(num)
Typer.appendToText("\n")
inputStr = num
StoryTracker.processInput()
}
}
function start(){
Typer.speed=1
Typer.text = ""
Typer.appendToText(start_text)
StoryTracker.getFirstStory()
startTyping()
console.log("Not mobile device");
document.getElementById('buttons').style.visibility='hidden';
}
$(document).ready(function() {
start()
})
-138
View File
@@ -1,138 +0,0 @@
start_text = "<span id='a'>Adventurer@AIDungeon</span>:<span id='b'>~</span><span id='c'>$</span> ./EnterDungeon <br/><!-- laglaglaglaglaglaglaglaglaglaglag-->"
function isMobileDevice() {
return /Android|webOS|iPhone|iPad|iPod|BlackBerry|IEMobile|Opera Mini/i.test(navigator.userAgent)
};
// Used to control the terminal like screen typing
var Typer={
text:null,
inputStr:"",
index:0,
speed:2,
acceptInput:false,
inputReady:false,
content:function(){
return $("#console").html()
},
appendToText:function(str){
str = str.replace(".", "." + "<!-- laglaglag-->")
typing = true
Typer.text = Typer.text + str;
},
removeChar:function(){
var cont=Typer.content()
$("#console").html($("#console").html().substring(0,cont.length-1))
Typer.text = Typer.text.substring(0, Typer.text.length-1)
Typer.index = Typer.index - 1
},
addText:function(){
if (Typer.index <= Typer.text.length) {
var cont=Typer.content()
if(cont.substring(cont.length-1,cont.length)=="|")
$("#console").html($("#console").html().substring(0,cont.length-1))
if (Typer.text.substring(Typer.index, Typer.index + Typer.speed).includes(".")){
Typer.index += 1
}
else{
Typer.index+=Typer.speed
}
var text=Typer.text.substring(0,Typer.index)
var rtn= new RegExp("\n", "g")
$("#console").html(text.replace(rtn,"<br/>"))
}
else{
typing = false
}
},
sendInput:function(){
request = Typer.inputStr
$.post("/generate", {action: request}, receiveResponse)
Typer.inputStr = ""
},
startTyping:function(){
addTextTimer = setInterval("Typer.addText()", 20)
},
KeyCheck:function(evt) {
evt = evt || window.event
var charCode = evt.keyCode || evt.which
if(charCode == 8){
if(Typer.inputStr.length > 0){
console.log(Typer.inputStr)
Typer.inputStr = Typer.inputStr.substring(0, Typer.inputStr.length-1)
console.log(Typer.inputStr)
Typer.removeChar()
}
}
},
onKeyPressFunc:function(evt) {
if(Typer.acceptInput && !isMobileDevice()){
evt = evt || window.event
var charCode = evt.keyCode || evt.which
if(charCode == 13){
Typer.acceptInput = false
Typer.sendInput()
}
else{
var charStr = String.fromCharCode(charCode)
Typer.appendToText(charStr)
Typer.inputStr = Typer.inputStr + charStr
}
}
},
}
function onButtonClick(num){
document.getElementById('buttons').style.visibility='hidden';
if (Typer.acceptInput == true){
Typer.acceptInput = false
num = String(num)
Typer.appendToText(num)
Typer.inputStr = num
Typer.sendInput
}
}
function receiveResponse(text){
Typer.appendToText(text)
Typer.acceptInput = true
}
function start(){
Typer.speed=1
Typer.text = ""
Typer.appendToText(start_text)
Typer.startTyping()
request_str = ""
$.post("/generate", {action: request_str}, receiveResponse)
document.getElementById('buttons').style.visibility='hidden';
}
document.onkeypress = Typer.onKeyPressFunc
document.addEventListener("keydown", Typer.KeyCheck);
$(document).ready(function() {
start()
})
-131
View File
@@ -1,131 +0,0 @@
body {
background-color: #000
}
#console {
font-family: courier, monospace;
color: #fff;
max-width: 800px;
width:80%;
margin-left:auto;
margin-right:auto;
margin-top:calc(30px + 4.0vh);
font-size:14px;
background-color: #000;
}
.about {
font-family: courier, monospace;
max-width: 800px;
width:80%;
margin-left:auto;
margin-right:auto;
margin-top:100px;
font-size:14px;
}
.about text{
color: #fff;
}
.about h2{
color: #0bc;
font-size:16px;
}
a {
color: #0bc;
text-decoration: none;
}
#a {
color: #0f0;
}
#c {
color: #0bc;
}
#b {
color: #ff0096;
}
#k {
animation: change 1s;
}
#op{
color: #888888
}
#SI{
background-color:transparent;
}
@keyframes change {
0% { color: #333; }
50% { color: #0f0; }
99% { color: black; }
}
#buttons {
position: relative;
bottom: 0px;
height: calc(100px + 12.0vw);
max-width: 800px;
width:80%;
color: 334;
background-color:#000;
margin-left:auto;
margin-right:auto;
display: flex;
}
button {
border-color: #fff;
color: #fff;
margin-top: 60px;
margin-left: 3.0vw;
margin-right: 3.0vw;
margin-bottom: 30px;
background-color: #111;
width: 14.0vw;
height: 12.0vw;
text-align: center;
border-radius: 25px;
font-size:30px;
}
button:focus{
background-color: #0bc;
}
/* Add a black background color to the top navigation */
.topnav {
background-color: #000;
max-width: 800px;
width:80%;
margin-left:auto;
margin-right:auto;
}
/* Style the links inside the navigation bar */
.topnav a {
font-family: courier, monospace;
display: inline-block;
color: #0bc;
text-align: center;
padding: 10px 5px;
text-decoration: none;
font-size:18px;
}
/* Change the color of links on hover */
.topnav a:hover {
background-color: #0bc;
color: white;
}
-53
View File
@@ -1,53 +0,0 @@
from flask import g
from flask import session
import os
from story.utils import *
import json
from flask import Flask, render_template, request, abort
from story.story_manager import *
from generator.web.web_generator import *
from other.cacher import *
import numpy as np
app = Flask(__name__)
app.secret_key = '#d\xe0\xd1\xfb\xee\xa4\xbb\xd0\xf0/e)\xb5g\xdd<`\xc7\xa5\xb0-\xb8d0S'
CRED_FILE = "./AI-Adventure-2bb65e3a4e2f.json"
# Bread and butter of app, updates story and returns based on choice
@app.route('/generate', methods=['POST'])
def generate():
action = request.form["action"]
# If there is no story in session, make a new one
if "prompt" not in session or session["prompt"] is None:
session["prompt"] = get_story_start("classic")
response = "Continue the initial story block:\n\n" + session["prompt"]
# If there is a story in session continue from it.
elif "story" not in session or session["story"] is None:
story_start = session["prompt"] + action
story = Story(story_start)
session["story"] = story.to_json()
response = "Enter the action then two newlines then the result:\n\n> "
else:
story = Story("")
story = story.initialize_from_json(session["story"])
action_result = action.split("\n")
story.add_to_story(action, result)
session["story"] = story.to_json()
response = "Enter the action then two newlines then the result:\n\n> "
print("Returning response")
return response
# Routes to index
@app.route('/')
def root():
session["story"] = None
return render_template('index.html')
if __name__ == '__main__':
app.run(host='0.0.0.0', port=8080)
-43
View File
@@ -1,43 +0,0 @@
<html>
<meta name="viewport" content="width=device-width, initial-scale=1">
<head>
<!-- Google Analytics -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-139423787-1', 'auto');
ga('send', 'pageview');
</script>
<!-- End Google Analytics -->
<title>About</title>
<link rel="stylesheet" type="text/css" href="static/style.css">
</head>
<div class="topnav">
<a href="../">Home</a>
<a href="http://patreon.com/AIDungeon">Support</a>
</div>
<div class="about">
<h2> About AI Dungeon </h2>
<text>
AI Dungeon is an AI generated text adventure that uses deep learning to create each adventure. It uses OpenAI's new <a href="https://openai.com/blog/better-language-models/">GPT-2 model</a>, which has 117 million parameters, to generate each story block and possible action.
<br><br> The first couple sentences of AIDungeon and the action verbs are handcrafted, but everything else is not. For each choice that is made, the initial prompt, the last story block, and the last action are fed into the neural network. The resulting story and action options are then output by the model.
<br><br> In order to speed up the experience some of the first sets of stories, choices and results have been pre-generated and cached. After enough choices, however, it will start taking longer (around 20s) to generate each result.
<br><br> As you can probably tell there's still a ways to go before AI will be your group's dungeon master, but even after running hundreds of adventures it still manages to surprise me in interesting ways. I've had a lot of fun making this and hope you enjoy it too.
<br><br> Warning: The GPT-2 model was trained on a huge amount of internet text so there might be offensive content.
<br><br> If you want to contact me about suggested improvements, comments, etc... feel free to email me about them at aidungeon.io@gmail.com.
<br><br> AI Dungeon was created by Nick Walton with the support of the <a href="https://pcc.cs.byu.edu/">BYU Percepton Cognition and Control Lab</a>, Alan Walton and Max Robinson.
</text>
</div>
</html>

Some files were not shown because too many files have changed in this diff Show More