Language Research

Research Scientists & Research Interns @ Language Research, NAVER AI Lab

Join Our Team!

About Us

About the Research Scientists

About the Research Interns

About Us

Language Research Team at NAVER AI Lab is dedicated to understanding humanity and society, and advancing human-like but also trustworthy and safe language models and Artificial Intelligence. As a team operating in both academic and industrial environments, we strive to tackle problems that are both fundamental and relevant to the real world.

Our current Research Mission and Interests are centered around building trustworthy and safe Large Language Models (LLMs), with a focus on:

• Datasets, Benchmarks, and Evaluation Metrics for LLMs

• LLM Security: Attacks, Defenses & Detections

• Safety Alignment, Learning, and Inference Algorithms

• LLM Agents, (Multi-)Agent Interactions, Decision-making, and Autonomous Agents

Check out our latest papers*(selected papers, all).*

---

About the Research Scientists

About the Role

We are looking for Research Scientists to join our team to research and development of safe and trustworthy Language Models and AI. The Research Scientists are encouraged to lead and/or support research projects collaboratively within the team and across the research field, other teams, and external organizations.

Specifically, the research topics include following but not limited to:

• Red-teaming, Adversarial Attack, Security Attack

• Watermarking

• Training Data/Privacy Probing & Leakage

• Model/Data/Task Contamination

• Robustness

• Safety Alignment

• Model Unlearning

• AI Explainability & Interpretability

• Causality

• Societal Impact by LLM Applications

Key Responsibilities

• Undertake pioneering research by formulating challenging research questions and devising problem-solving methods.

• Lead a wide range of research activities including but not limited to the ideation and development of safe and trustworthy AI systems, and authoring research papers.

• Communicate research progress and findings clearly and effectively.

• Actively collaborate with other researchers.

• Report and present the research findings and developments at top-tier academic venues.

Requirements

• Holds a PhD degree or equivalent (or expected to receive within 6 months) in Computer Science (CS), Electrical Engineering (EE), Mathematics, or other relevant fields.

• An academic publication record at top-tier conferences in Natural Language Processing (e.g., *ACL), Machine Learning (e.g., NeurIPS, ICLR), and others (e.g., FAccT).

• Experience in research collaborations and academic writing in related fields.

  • (Preferred) Global research/industrial collaboration experiences.

• Excellent analytical and problem-solving skills.

• Strong communication skills, openness to constructive discussion, and receptiveness to feedback.

How to apply

• Please apply at here (Korean/ English) (sign-in required)

  • Application category: Tech > Common > Common > AI Safety (Full-time)

• Hiring process

  • Application screening → Coding test → Job talk → Interview → (optional) Second Interview → Notification

---

About the Research Interns

About the Internship

Our team is offering research intern positions for 2024 Fall and 2025 Winter. As an intern, you'll be actively involved in developing and conducting research on trustworthy and safe large language models.

Before starting your internship, we will discuss closely to refine and develop your research plan. This process ensures that your proposal aligns with our mutual research interests. We strongly support your initiative to lead your main project while also engaging in other research projects. This approach offers a balanced experience in both research leadership and collaboration.

A key goal of this internship is to produce academic papers suitable for submission to top-tier conferences or journals. Additionally, we anticipate that the outcomes of the project will make meaningful contributions to real-world applications.

• This is a full-time, in-person role at NAVER 1784 (Seongnam-si, Gyeonggi-do, South Korea)

  • The office could be changed to NAVER Green Factory or the other near building.

• This internship offers a flexible starting date.

Key Responsibilities

• Undertake pioneering research by formulating challenging research questions and devising problem-solving methods. This includes implementing and evaluating models, as well as authoring research papers.

• Communicate research progress and findings clearly and effectively.

• Demonstrate proactivity and the ability to successfully complete projects.

Requirements

• Pursuing a PhD or equivalent in Computer Science (CS), Electrical Engineering (EE), Mathematics, or other relevant fields.

• At least one paper authored as the first author in AI/ML-related conferences.

  • (Preferred) A strong academic publication record at top-tier conferences in Natural Language Processing (e.g., *CL), Machine Learning (e.g., NeurIPS, ICLR), and others.

• Experience in research collaborations and academic writing in related fields.

• Excellent analytical and problem-solving skills.

• Strong communication skills, openness to constructive discussion, and receptiveness to feedback.`

How to apply

• Please apply here (Korean/English) (sign-in required)

  • Application category: Tech > Common > Common > AI (Intern)

• Your application should include the following:

  • CV

  • Brief research interests and research plans

    • that include research questions and goals with a few related works.

    • that include brief idea and direction to solve the problem. (not necessary to be perfect!)

• Hiring process

  • Application screening → Coding test → Job talk → Interview → (optional) Second Interview → Notification

• Note

  • Please submit your application by Sept. 6.

  • This position could be closed early when the position is full.

We look forward to your application and the possibility of you joining our team. If you have any question, please contact us! 🤗

---

Selected Papers

• MAQA: Evaluating Uncertainty Quantification in LLMs Regarding Data Uncertainty, Yongjin Yang, Haneul Yoo, Hwaran Lee, Arxiv, 2024 dataset & benchmarkuncertainty
• CSRT: Evaluation and Analysis of LLMs using Code-Switching Red-Teaming Dataset, Haneul Yoo, Yongjin Yang, Hwaran Lee, Arxiv, 2024 dataset & benchmark llm-security

• AdvisorQA: Towards Helpful and Harmless Advice-seeking Question Answering with Collective Intelligence , Minbeom Kim, Hwanhee Lee, Joonsuk Park, Hwaran Lee, Kyomin Jung, Arxiv, 2024 alignment dataset & benchmark

• Who Wrote this Code? Watermarking for Code Generation, Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, Gunhee Kim, ACL, 2024 llm-security

• Calibrating Large Language Models Using Their Generations Only, Dennis Thomas Ulmer, Martin Gubri, Hwaran Lee, Sangdoo Yun, Seong Joon Oh, ACL, 2024 uncertainty

• TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identification, Martin Gubri, Dennis Thomas Ulmer, Hwaran Lee, Sangdoo Yun, Seong Joon Oh, ACL Findings, 2024 llm-security

• KorNAT: LLM Alignment Benchmark for Korean Social Values and Common Knowledge, Jiyoung Lee, Minwoo Kim, Seungho Kim, Junghwan Kim, Seunghyun Won, Hwaran Lee, Edward Choi, ACL Findings, 2024 dataset & benchmark

• TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models, Jaewoo Ahn, Taehyun Lee, Junyoung Lim, Jin-Hwa Kim, Sangdoo Yun, Hwaran Lee, Gunhee Kim, ACL Findings, 2024 dataset & benchmark

• LifeTox: Unveiling Implicit Toxicity in Life Advice, M Kim, J Koo, H Lee, J Park, H Lee, K Jung, NAACL (Shrot) dataset & benchmark

• Prometheus: Inducing Fine-grained Evaluation Capability in Language Models, S Kim, J Shin, Y Cho, J Jang, S Longpre, H Lee, S Yun, S Shin, S Kim, J Throne, M Seo, ICLR 2024 dataset evaluation

• EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria, TS Kim, Y Lee, J Shin, YH Kim, J Kim, arXiv preprint arXiv:2309.13633 evaluation

• KoBBQ: Korean Bias Benchmark for Question Answering, J Jin, J Kim, N Lee, H Yoo, A Oh, H Lee, TACL dataset & benchmark

• Revealing User Familiarity Bias in Task-Oriented Dialogue via Interactive Evaluation, T Kim, J Shin, YH Kim, S Bae, S Kim, arXiv preprint arXiv:2305.13857 evaluation

• The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning, S Kim, SJ Joo, D Kim, J Jang, S Ye, J Shin, M Seo, EMNLP 2023 dataset

• Aligning Large Language Models through Synthetic Feedback, S Kim, S Bae, J Shin, S Kang, D Kwak, KM Yoo, M Seo, EMNLP 2023 alignment

• ProPILE: Probing Privacy Leakage in Large Language Models, S Kim, S Yun, H Lee, M Gubri, S Yoon, SJ Oh, NeurIPS 2023 (spotlight) llm-security

• Who Wrote this Code? Watermarking for Code Generation, T Lee, S Hong, J Ahn, I Hong, H Lee, S Yun, J Shin, G Kim, arXiv preprint arXiv:305.15060 llm-security

• KoSBi: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Application, H Lee, S Hong, J Park, T Kim, G Kim, JW Ha, ACL 2023 (industry track) dataset & benchmark

• SQuARe: A Large-Scale Dataset of Sensitive Questions and Acceptable Responses Created Through Human-Machine Collaboration, H Lee, S Hong, J Park, T Kim, M Cha, Y Choi, BP Kim, G Kim, EJ Lee, Y Lim, A Oh, S Park, JW Ha, ACL 2023 (best paper nominated) dataset & benchmark

• Query-Efficient Black-Box Red Teaming via Bayesian Optimization, D Lee, JY Lee, JW Ha, JH Kim, SW Lee, H Lee, HO Song, ACL 2023 llm-security

• Critic-Guided Decoding for Controlled Text Generation, M Kim, H Lee, KM Yoo, J Park, H Lee, K Jung, ACL 2023 (Findings) learning & inference

• ClaimDiff: Comparing and Contrasting Claims on Contentious Issues, M Ko, I Seong, H Lee, J Park, M Chang, M Seo, ACL 2023(Findings) dataset & benchmark