MSCI 641 Assignment 4

接下来，我会发题给你，你必须阅读题目到最后一行，再开始带我解答这个题目。你必须认真读完我发给你的全部内容，然后解答题目：
Write a python script using PyTorch/Keras to train a fully-connected feed-forward neural network classifier to classify documents in the Amazon corpus into positive and negative classes. Your network must consist of:

1. Input layer of the word2vec embeddings you prepared in Assignment 3.
2. One hidden layer. For the hidden layer, try the following activation functions: ReLU, sigmoid and tanh.
3. Final layer with softmax activation function.
4. Use cross-entropy as the loss function.
5. Add L2-norm regularization.
6. Add dropout. Try a few different dropout rates.

For this assignment, you must use your training/validation/test data splits from Assignment 1. Train your models on the **training** set. You may only tune your models on your **validation** set. 

Once the development is complete, run your classifier on your **test** set.

Report your classification accuracy results in a table with three different activation functions in the hidden layer (ReLU, sigmoid and tanh). What effect do activation functions have on your results? What effect does addition of L2-norm regularization have on the results? What effect does dropout have on the results? Explain your intuitions briefly (up to 10 sentences).
我作业三的代码如下：
```python
from gensim.models import Word2Vec  # Gensim中的Word2Vec模型
from gensim.models.word2vec import LineSentence  # Gensim中用于处理文本数据的工具
import time, string

start_time = time.time()


def remove_punctuation(word_list):
    return [word.translate(str.maketrans('', '', string.punctuation)).lower() for word in word_list]


# 定义训练模型的函数，需要三个参数：正面评论的文件路径、负面评论的文件路径、保存模型的文件路径
def train_model(pos_file_path, neg_file_path, model_file_path):
    # 使用 with open 来打开文件，这样可以保证文件在用完后会被正确关闭
    with open(pos_file_path, 'r') as pos_file, open(neg_file_path, 'r') as neg_file:
        # 使用 LineSentence 处理文件中的每一行文本，并将处理结果转为列表，然后将正面和负面评论的列表连接起来
        sentences_pos = [remove_punctuation(sentence) for sentence in LineSentence(pos_file)]
        sentences_neg = [remove_punctuation(sentence) for sentence in LineSentence(neg_file)]
        sentences = sentences_pos + sentences_neg
        # 使用连接起来的句子列表创建Word2Vec模型，设置向量大小为100，窗口大小为5，最小词频为1，使用4个工作线程进行训练
        model = Word2Vec(sentences=sentences, vector_size=100, window=5, min_count=1, workers=4)
        # 保存训练好的模型到指定的文件路径
        model.save(model_file_path)


# Python的入口点，只有直接运行这个脚本时，下面的代码才会被执行
if __name__ == "__main__":
    # 正面评论的文件路径
    pos_file_path = 'data/pos.txt'
    # 负面评论的文件路径
    neg_file_path = 'data/neg.txt'
    # 保存模型的文件路径
    model_file_path = 'w2v.model'
    # 调用上面定义的函数进行模型训练
    train_model(pos_file_path, neg_file_path, model_file_path)
    print("time:>>>", time.time() - start_time)
```
额外要求：

必须使用 HW1 中输出的训练、验证和测试集来训练模型，只能用训练集进行训练，验证集进行调试，并用测试集来比对准确率， 可以使用带 stopword 或不带 stopword 的集。（即带_ns和不带_ns的）哪个给出的准确率更高用哪个

输出三个模型，分别是
nn_relu.model，使用RELU作为激活函数
nn_sigmoid.model，使用sigmoid作为激活函数
nn_tanh.model，使用tanh作为激活函数

建立一个inference.py文件，需要接受两个命令行指令
1：路径到一个.txt文件，文件中包含用于测试的句子，每行一句
2：使用的分类器类型，暨输出的三个模型之一

现在所拥有的文件以及部分数据：
训练集：train.csv、train_ns.csv
验证集：val_ns.csv、val.csv
测试集：test.csv、test_ns.csv
其中_ns后缀代表内容已经移除 stopword。
注意1：我的所有数据文件已经实现分词，并且第一列都标记了positive或negative。其他列都是分好的词。
注意2：不要假设已经将数据加载到了 pandas DataFrame 中，并且 DataFrame 的列为 'label' 和 'text'。切记我并没有提前完成，我其他的也是，全部没有提前完成。
接下来我给你提供每个文件的部分数据。
train_ns.csv 部分数据如下：
```csv
positive,little,dude,going,running,every,day,It's,great,holds,phone,securely,,,,,,,,,,,,,,
positive,don't,know,took,long,get,one,,,,,,,,,,,,,,,,,,,
negative,cord,cheap,wires,coming,plastic,using,2,DAYS,,,,,,,,,,,,,,,,,
positive,might,want,look,lineup,want,capable,holding,,,,,,,,,,,,,,,,,,
negative,It's,pretty,small,even,big,enough,either,bigger,cats,,,,,,,,,,,,,,,,
positive,remove,app,screens,,,,,,,,,,,,,,,,,,,,,,
```
train.csv 部分数据如下：
```csv
positive,I,went,through,two,others,before,finding,this,one,,,,,,,,,,,,,,,,
positive,I,was,doubtful,that,anything,could,survive,the,tugging,and,pulling,of,his,use,and,the,weight,of,his,jackets,,,,,
positive,She,watched,the,accompanying,DVD,video,and,immediately,set,out,on,a,grocery,shopping,spree,and,began,to,use,this,fantastic,machine,,,
```
val_ns.csv 部分数据如下：
```csv
negative,Actually,don't,need,many,given,small,overall,size,,,,,,,,,,,,,,,,,
positive,do't,drink,coffee,like,cleaning,glass,shards,glass,pot,broke,,,,,,,,,,,,,,,
positive,easy,scoop,cookie,dough,uniform,sizes,goopy,fingers,trying,get,dough,spoon,,,,,,,,,,,,,
```
val.csv 部分数据如下：
```csv
negative,Very,hard,to,adjust,the,volume,when,the,static,begins,,,,,,,,,,,,,,,
positive,Why?,Because,I,have,been,cutting,fruit,for,so,long,that,I,can,do,it,practically,blindfolded,,,,,,,,
negative,But,I,want,a,boar,bristle,brush,not,a,everyday,brushI,was,expecting,great,things,from,my,first,boar,hair,ionic,bristle,combo,brush,
```
test_ns.csv 部分数据如下：
```csv
negative,anxious,try,looking,forward,taking,,,,,,,,,,,,,,,,,
positive,older,KitchenAid,didn't,feel,could,justify,pricey,upgrade,,,,,,,,,,,,,,
positive,friend,gave,hersbut,could,find,guidance,box,,,,,,,,,,,,,,,
```

test.csv 部分数据如下：
```csv
negative,You,have,to,remove,the,case,every,time,you,charge,the,phone,,,,,,,,,,,,,
positive,For,the,1299,it's,a,solid,deal,and,gives,you,3,mixing,bowls,that,will,last,you,a,long,time,,,,,
negative,Had,I,received,the,item,I,ordered,the,review,would,be,higher,,,,,,,,,,,,,
```

接下来，我会发题给你，你必须阅读题目到最后一行，再开始带我解答这个题目。你必须认真读完我发给你的全部内容，然后解答题目：
Write a python script using PyTorch/Keras to train a fully-connected feed-forward neural network classifier to classify documents in the Amazon corpus into positive and negative classes. Your network must consist of:

1. Input layer of the word2vec embeddings you prepared in Assignment 3.
2. One hidden layer. For the hidden layer, try the following activation functions: ReLU, sigmoid and tanh.
3. Final layer with softmax activation function.
4. Use cross-entropy as the loss function.
5. Add L2-norm regularization.
6. Add dropout. Try a few different dropout rates.

For this assignment, you must use your training/validation/test data splits from Assignment 1. Train your models on the **training** set. You may only tune your models on your **validation** set. 

Once the development is complete, run your classifier on your **test** set.

Report your classification accuracy results in a table with three different activation functions in the hidden layer (ReLU, sigmoid and tanh). What effect do activation functions have on your results? What effect does addition of L2-norm regularization have on the results? What effect does dropout have on the results? Explain your intuitions briefly (up to 10 sentences).
我作业三训练的模型名称是：w2v.model

额外要求：
必须使用 HW1 中输出的训练、验证和测试集来训练模型，只能用训练集进行训练，验证集进行调试，并用测试集来比对准确率， 可以使用带 stopword 或不带 stopword 的集。（即带_ns和不带_ns的）哪个给出的准确率更高用哪个

输出三个模型，分别是
nn_relu.model，使用RELU作为激活函数
nn_sigmoid.model，使用sigmoid作为激活函数
nn_tanh.model，使用tanh作为激活函数

建立一个inference.py文件，需要接受两个命令行指令
1：路径到一个.txt文件，文件中包含用于测试的句子，每行一句
2：使用的分类器类型，暨输出的三个模型之一

现在所拥有的文件以及部分数据：
训练集：train.csv、train_ns.csv
验证集：val_ns.csv、val.csv
测试集：test.csv、test_ns.csv
其中_ns后缀代表内容已经移除 stopword。
注意1：我的所有数据文件已经实现分词，并且第一列都标记了positive或negative。其他列都是分好的词。
注意2：不要假设已经将数据加载到了 pandas DataFrame 中，并且 DataFrame 的列为 'label' 和 'text'。切记我并没有提前完成，我其他的也是，全部没有提前完成。
接下来我给你提供每个文件的部分数据。
train_ns.csv 部分数据如下：
```csv
positive,little,dude,going,running,every,day,It's,great,holds,phone,securely,,,,,,,,,,,,,,
positive,don't,know,took,long,get,one,,,,,,,,,,,,,,,,,,,
negative,cord,cheap,wires,coming,plastic,using,2,DAYS,,,,,,,,,,,,,,,,,
positive,might,want,look,lineup,want,capable,holding,,,,,,,,,,,,,,,,,,
negative,It's,pretty,small,even,big,enough,either,bigger,cats,,,,,,,,,,,,,,,,
positive,remove,app,screens,,,,,,,,,,,,,,,,,,,,,,
```
train.csv 部分数据如下：
```csv
positive,I,went,through,two,others,before,finding,this,one,,,,,,,,,,,,,,,,
positive,I,was,doubtful,that,anything,could,survive,the,tugging,and,pulling,of,his,use,and,the,weight,of,his,jackets,,,,,
positive,She,watched,the,accompanying,DVD,video,and,immediately,set,out,on,a,grocery,shopping,spree,and,began,to,use,this,fantastic,machine,,,
```
val_ns.csv 部分数据如下：
```csv
negative,Actually,don't,need,many,given,small,overall,size,,,,,,,,,,,,,,,,,
positive,do't,drink,coffee,like,cleaning,glass,shards,glass,pot,broke,,,,,,,,,,,,,,,
positive,easy,scoop,cookie,dough,uniform,sizes,goopy,fingers,trying,get,dough,spoon,,,,,,,,,,,,,
```
val.csv 部分数据如下：
```csv
negative,Very,hard,to,adjust,the,volume,when,the,static,begins,,,,,,,,,,,,,,,
positive,Why?,Because,I,have,been,cutting,fruit,for,so,long,that,I,can,do,it,practically,blindfolded,,,,,,,,
negative,But,I,want,a,boar,bristle,brush,not,a,everyday,brushI,was,expecting,great,things,from,my,first,boar,hair,ionic,bristle,combo,brush,
```
test_ns.csv 部分数据如下：
```csv
negative,anxious,try,looking,forward,taking,,,,,,,,,,,,,,,,,
positive,older,KitchenAid,didn't,feel,could,justify,pricey,upgrade,,,,,,,,,,,,,,
positive,friend,gave,hersbut,could,find,guidance,box,,,,,,,,,,,,,,,
```

test.csv 部分数据如下：
```csv
negative,You,have,to,remove,the,case,every,time,you,charge,the,phone,,,,,,,,,,,,,
positive,For,the,1299,it's,a,solid,deal,and,gives,you,3,mixing,bowls,that,will,last,you,a,long,time,,,,,
negative,Had,I,received,the,item,I,ordered,the,review,would,be,higher,,,,,,,,,,,,,
```