MSCI 641 Assignment Total

我的题目如下：
Write a python script to perform the following data preparation activities:

1. Tokenize the corpus
2. Remove the following special characters: !"#$%&()*+/:;<=>@[\\]^`{|}~\t\n
3. Create two versions of your dataset: (1) with stopwords and (2) without stopwords. Stopword lists are available online.
4. Randomly split your data into training (80%), validation (10%) and test (10%) sets.
Details:

1. **Command-line argument:** Path to the folder where pos.txt and neg.txt reside. 
2. You must implement the required techniques in core Python. For instance, you can use Python’s `random` module to create the train/val/test split and can read the .txt files in Python (using the in-built `open` function) instead of using `pandas`. 
3. The tokenized sentences should be stored in a file called ‘out.csv’ where each line follows the format mentioned below. Note that the quotes below are just to show that they are strings, you don’t need to explicitly add them around every token: [‘My’, ‘daughter’, ‘wanted’, ‘this’, ‘book’, ‘and’, ‘the’, ‘price’, ‘on’, ‘Amazon’, ‘was’, ‘the’, ‘best.’] 
4. Although not required, you can remove any additional punctuation marks such as ‘-’.  Additionally, it doesn’t matter if you store those punctuations as a separate token or concatenate it with the previous word. 
5. **Expected output files:** 
    1. out.csv: tokenized sentences w/ stopwords 
    2. train.csv: training set w/ stopwords 
    3. val.csv: validation set w/ stopwords 
    4. test.csv: test set w/ stopwords 
    5. out_ns.csv: tokenized sentences w/o stopwords 
    6. train_ns.csv: training set w/o stopwords 
    7. val_ns.csv: validation set w/o stopwords 
    8. test_ns.csv: test set w/o stopwords
6. It is recommended that you generate the labels as well in this assignment even though it is not mandatory
我有两个文件，分别为：neg.txt、pos.txt。
neg.txt 的部分数据和格式如下：
```txt
I bought this when I bought the pop maker.
As for "pop embellishing" well, that wasn't too hard to figure out, either.
I'd save the money and spend it instead on extra pop sticks, which seem to disappear the way socks do.
didn't really care many of the cakes at all.
not up to normal standing for wilton yearbooks of the past.
Buy a Wilton magazine for less money and get more ideas and instructions for your investment.
Bag tore with almost nothing in it - Just caught the corner of a small cracker box and that was that.
```
pos.txt
```txt
My daughter wanted this book and the price on Amazon was the best.
She has already tried one recipe a day after receiving the book.
I bought this zoku quick pop for my daughterr with her zoku quick maker.
She loves it and have fun to make her own ice cream.
I was hoping there were more where those came from.
This book emphasizes very sweet dessert pops, however.
```

接下来，我会发题给你，你必须阅读题目到最后一行，再开始带我解答这个题目。你必须认真读完我发给你的全部内容，然后解答题目：
Write a python script using PyTorch/Keras to train a fully-connected feed-forward neural network classifier to classify documents in the Amazon corpus into positive and negative classes. Your network must consist of:

1. Input layer of the word2vec embeddings you prepared in Assignment 3.
2. One hidden layer. For the hidden layer, try the following activation functions: ReLU, sigmoid and tanh.
3. Final layer with softmax activation function.
4. Use cross-entropy as the loss function.
5. Add L2-norm regularization.
6. Add dropout. Try a few different dropout rates.

For this assignment, you must use your training/validation/test data splits from Assignment 1. Train your models on the **training** set. You may only tune your models on your **validation** set. 

Once the development is complete, run your classifier on your **test** set.

Report your classification accuracy results in a table with three different activation functions in the hidden layer (ReLU, sigmoid and tanh). What effect do activation functions have on your results? What effect does addition of L2-norm regularization have on the results? What effect does dropout have on the results? Explain your intuitions briefly (up to 10 sentences).
我作业三训练的模型名称是：w2v.model

额外要求：
必须使用 HW1 中输出的训练、验证和测试集来训练模型，只能用训练集进行训练，验证集进行调试，并用测试集来比对准确率， 可以使用带 stopword 或不带 stopword 的集。（即带_ns和不带_ns的）哪个给出的准确率更高用哪个

输出三个模型，分别是
nn_relu.model，使用RELU作为激活函数
nn_sigmoid.model，使用sigmoid作为激活函数
nn_tanh.model，使用tanh作为激活函数

建立一个inference.py文件，需要接受两个命令行指令
1：路径到一个.txt文件，文件中包含用于测试的句子，每行一句
2：使用的分类器类型，暨输出的三个模型之一

现在所拥有的文件以及部分数据：
我作业3训练的模型名称是：w2v.model
训练集：out_train.csv、out_ns_train.csv
验证集：out_val.csv、out_ns_val.csv
测试集：out_test.csv、out_ns_test.csv
其中包含ns的代表内容已经移除 stopword。
所有数据的结构如下：第一列是标签、第二例是文本
```csv
label,text
neg,I had to take an antihistamine to calm the itching
neg,The bottle is lovely and very girlie I love the amethyst jewel tone of purple
neg,When I got my new computer I was not at all happy with the USB keyboard that came with it
neg,I will get this brand of Halvah.has too many artificial ingredients
neg,It lingers forever long after you 've finished your last sip and it 's not very pleasant
neg,This stuff was so fine it went right through my strainer when I tried to minimize the particles
neg,I bought this cable and it does n't work
```

我的题目如下：
Write a python script using PyTorch/Keras to train a fully-connected feed-forward neural network classifier to classify documents in the Amazon corpus into positive and negative classes. Your network must consist of: 

1. Input layer of the word2vec embeddings you prepared in Assignment 3.
2. One hidden layer. For the hidden layer, try the following activation functions: ReLU, sigmoid and tanh.
3. Final layer with softmax activation function.
4. Use cross-entropy as the loss function.
5. Add L2-norm regularization.
6. Add dropout. Try a few different dropout rates.

For this assignment, you must use your training/validation/test data splits from Assignment 1. Train your models on the **training** set. You may only tune your models on your **validation** set. Once the development is complete, run your classifier on your **test** set.

Report your classification accuracy results in a table with three different activation functions in the hidden layer (ReLU, sigmoid and tanh). What effect do activation functions have on your results? What effect does addition of L2-norm regularization have on the results? What effect does dropout have on the results? Explain your intuitions briefly (up to 10 sentences).

1. DO NOT tune your models on the **test** set. 
2. You need to submit only **one** model that is trained on either with or without stopwords version (whichever gives you better performance). 

3. **Output files (must be included in your submission):**
    1. nn_relu.model: Classifier w/ ReLU activation
    2. nn_sigmoid.model: Classifier w/ Sigmoid activation 
    3. nn_tanh.model: Classifier w/ Tanh activation
4. **Code requirement:** 
    1. In addition to the general instructions posted at the beginning of the document, you should also submit an `inference.py` script, which classifies a given sentence into a positive/negative class. It should accept the **two** command-line arguments described below. 
        1. arg1: Path to a .txt file, which contains some sentences compiled for evaluation. There will be one sentence per line. 
        2. arg2: Type of classifier to use. Its value will be one of the following – relu, sigmoid, tanh. **Example:** relu indicates that the neural network with ReLU activation should be selected for classifying sentences in the aforementioned .txt file. 
现在所拥有的文件以及部分数据：
训练集：out_train.csv、out_ns_train.csv
验证集：out_val.csv、out_ns_val.csv
测试集：out_test.csv、out_ns_test.csv
其中包含ns的代表内容已经移除 stopword。
所有文件的数格式如下：
```csv
label,text
neg,I had to take an antihistamine to calm the itching
neg,The bottle is lovely and very girlie I love the amethyst jewel tone of purple
neg,When I got my new computer I was not at all happy with the USB keyboard that came with it
neg,I will get this brand of Halvah.has too many artificial ingredients
neg,It lingers forever long after you 've finished your last sip and it 's not very pleasant
neg,This stuff was so fine it went right through my strainer when I tried to minimize the particles
neg,I bought this cable and it does n't work
```
作业三的代码如下：
# -*- coding: utf-8 -*-
# @Time    : 2023/7/3 17:24
# @Author  : AI悦创
# @FileName: generate.py
# @Software: PyCharm
# @Blog    ：https://bornforthis.cn/
import argparse
import gensim
from nltk.tokenize import word_tokenize
from nltk.corpus import stopwords
import string


def read_corpus(pos_path, neg_path):
    stop_words = set(stopwords.words('english'))

    def tokenize_and_remove_stopwords(file_path):
        with open(file_path, 'r', encoding='utf-8') as file:
            for line in file:
                tokens = word_tokenize(line)
                yield [token.lower() for token in tokens if
                       token not in string.punctuation and token.lower() not in stop_words]

    return list(tokenize_and_remove_stopwords(pos_path)) + list(tokenize_and_remove_stopwords(neg_path))


def main():
    parser = argparse.ArgumentParser(description='Train Word2Vec on Amazon Corpus')
    parser.add_argument('--folder_path', type=str, help='Path to the folder where pos.txt and neg.txt reside.')
    args = parser.parse_args()

    corpus = read_corpus(args.folder_path + "/pos.txt", args.folder_path + "/neg.txt")

    model = gensim.models.Word2Vec(sentences=corpus, vector_size=100, window=5, min_count=1, workers=4)
    model.save(args.folder_path + "/w2v.model")


if __name__ == "__main__":
    main()

接下来，我会发题给你，你必须阅读题目到最后一行，再开始带我解答这个题目。你必须认真读完我发给你的全部内容，然后解答题目：
Write a python script using PyTorch/Keras to train a fully-connected feed-forward neural network classifier to classify documents in the Amazon corpus into positive and negative classes. Your network must consist of:

1. Input layer of the word2vec embeddings you prepared in Assignment 3.
2. One hidden layer. For the hidden layer, try the following activation functions: ReLU, sigmoid and tanh.
3. Final layer with softmax activation function.
4. Use cross-entropy as the loss function.
5. Add L2-norm regularization.
6. Add dropout. Try a few different dropout rates.

For this assignment, you must use your training/validation/test data splits from Assignment 1. Train your models on the **training** set. You may only tune your models on your **validation** set. 

Once the development is complete, run your classifier on your **test** set.

Report your classification accuracy results in a table with three different activation functions in the hidden layer (ReLU, sigmoid and tanh). What effect do activation functions have on your results? What effect does addition of L2-norm regularization have on the results? What effect does dropout have on the results? Explain your intuitions briefly (up to 10 sentences).
我作业三的代码如下：
```python
import argparse
import gensim
from nltk.tokenize import word_tokenize
from nltk.corpus import stopwords
import string


def read_corpus(pos_path, neg_path):
    stop_words = set(stopwords.words('english'))

    def tokenize_and_remove_stopwords(file_path):
        with open(file_path, 'r', encoding='utf-8') as file:
            for line in file:
                tokens = word_tokenize(line)
                yield [token.lower() for token in tokens if
                       token not in string.punctuation and token.lower() not in stop_words]

    return list(tokenize_and_remove_stopwords(pos_path)) + list(tokenize_and_remove_stopwords(neg_path))


def main():
    parser = argparse.ArgumentParser(description='Train Word2Vec on Amazon Corpus')
    parser.add_argument('--folder_path', type=str, help='Path to the folder where pos.txt and neg.txt reside.')
    args = parser.parse_args()

    corpus = read_corpus(args.folder_path + "/pos.txt", args.folder_path + "/neg.txt")

    model = gensim.models.Word2Vec(sentences=corpus, vector_size=100, window=5, min_count=1, workers=4)
    model.save(args.folder_path + "/w2v.model")


if __name__ == "__main__":
    main()

```
额外要求：

必须使用 HW1 中输出的训练、验证和测试集来训练模型，只能用训练集进行训练，验证集进行调试，并用测试集来比对准确率， 可以使用带 stopword 或不带 stopword 的集。（即带_ns和不带_ns的）哪个给出的准确率更高用哪个

输出三个模型，分别是
nn_relu.model，使用RELU作为激活函数
nn_sigmoid.model，使用sigmoid作为激活函数
nn_tanh.model，使用tanh作为激活函数

建立一个inference.py文件，需要接受两个命令行指令
1：路径到一个.txt文件，文件中包含用于测试的句子，每行一句
2：使用的分类器类型，暨输出的三个模型之一

现在所拥有的文件以及部分数据：
训练集：train.csv、train_ns.csv
验证集：val_ns.csv、val.csv
测试集：test.csv、test_ns.csv
其中_ns后缀代表内容已经移除 stopword。
注意1：我所有的数据都有 label和text标签
注意2：不要假设已经将数据加载到了 pandas DataFrame 中，并且 DataFrame 的列为 'label' 和 'text'。切记我并没有提前完成，我其他的也是，全部没有提前完成。
接下来我给你提供每个文件的部分数据。
train_ns.csv 部分数据如下：
```csv
label,text
neg,This item worked first day 's
neg,You probably enjoy reading book much playing game
pos,This reason I gave 4 instead 5 stars
pos,I dont actually use bake cookies fondant cutters work great
pos,I super excited get start using
pos,My husband mechanic dropped case prevents damage phone
neg,What n't care flavor ... lack thereof
```
train.csv 部分数据如下：
```csv
label,text
pos,The con price I still think well worth money
neg,The case slack phone well making good fit impossible
neg,It took get two tubes suspiciously resemble couple little dildos box
neg,That drive three would recognize tracks therefore would recognize disc
neg,Soon spray 've got color The fumes arent harsh still kinda stinks
pos,I needed small tongs serving carrot celery sticks dip
```
val_ns.csv 部分数据如下：
```csv
label,text
pos,The inside soft material protect screen
neg,The first one I purchased year ago made differently item sold
neg,In general son enjoy much
neg,Have already said game easy First time playing ranking A
neg,In order work I put bake broil burns outside cook inside
pos,worth wait wo n't recharge
```
val.csv 部分数据如下：
```csv
negative,Very,hard,to,adjust,the,volume,when,the,static,begins,,,,,,,,,,,,,,,
positive,Why?,Because,I,have,been,cutting,fruit,for,so,long,that,I,can,do,it,practically,blindfolded,,,,,,,,
negative,But,I,want,a,boar,bristle,brush,not,a,everyday,brushI,was,expecting,great,things,from,my,first,boar,hair,ionic,bristle,combo,brush,
```
test_ns.csv 部分数据如下：
```csv
label,text
neg,I had to take an antihistamine to calm the itching
neg,The bottle is lovely and very girlie I love the amethyst jewel tone of purple
neg,When I got my new computer I was not at all happy with the USB keyboard that came with it
neg,I will get this brand of Halvah.has too many artificial ingredients
neg,It lingers forever long after you 've finished your last sip and it 's not very pleasant
neg,This stuff was so fine it went right through my strainer when I tried to minimize the particles
neg,I bought this cable and it does n't work
```

test.csv 部分数据如下：
```csv
label,text
pos,It covers all the buttons power camera volume/zoom and menu .4
neg,If you do n't know what your doing it wo n't go on right
neg,Now I have two giant cans that I probably wo n't drink
neg,You will be flipping left and right going up and down with no sence of direction
```