-
Notifications
You must be signed in to change notification settings - Fork 39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
关于NER模型高层部分构成的问题 #29
Comments
交叉熵主要是针对不使用CRF的,可参考:https://github.com/taishan1994/pytorch_bert_bilstm_crf_ner |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
作者您好,您的项目对我有很大的帮助,但是关于NER模型高层部分构成的目的我有一些问题想要请教您。
![image](https://private-user-images.githubusercontent.com/83754033/238170856-b7952072-7261-4ffd-a064-870ab7c9f70a.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjUyOTM4OTMsIm5iZiI6MTcyNTI5MzU5MywicGF0aCI6Ii84Mzc1NDAzMy8yMzgxNzA4NTYtYjc5NTIwNzItNzI2MS00ZmZkLWEwNjQtODcwYWI3YzlmNzBhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MDIlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTAyVDE2MTMxM1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTZiM2Q2NDFjMGM3MmU5N2Y4NDc4Njk4ZDYwMTIwZDg4ZTAxMTNlYjE1Mzg0YzY0N2NmNTFkYzYzMmUyYWNkYzEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.QCkNFzREYEcvxuBzyaj4VrCGhJSNb1GfhXRmMQV2nU4)
首先下图是模型高层部分的构成图:
其中最后线性分类器的输出维度和CRF部分的tag数量我根据自己任务进行了更改,但这不会影响到我下面的问题。
我的问题是:
1、中间线性层中为何首先将768降维到256?256这个数字是通过什么方式得到的?
2、后续的ReLU激活函数的目的又是什么?
3、在模型真正实现的过程中dropout操作并没有真正采用,原因是否是因为在中间线性层中已经进行过了dropout?
期待您拨冗回复!
The text was updated successfully, but these errors were encountered: