# Backbone Research

### **About Us**

Our team is devoted to a comprehensive exploration of fundamental topics within the fields of deep learning and artificial intelligence. We have a strong passion for developing efficient and powerful models for deep neural networks and rigorously training them to establish solid foundation models. Initially focusing on computer vision and machine learning, we have recently broadened our scope to include language modeling and multimodal learning. Our research interests are summarized as follows:

### Research topics&#x20;

* **New Foundation Models/Elements:** We focus on modeling foundational architecture itself and underlying architectural elements in language/vision/vision language models.
* **New Training Methods:** Our team is also developing novel training methodologies that enhance the learning efficiency and effectiveness of our models.
* **Cost-efficiency Improvements:** We explore various avenues to increase overall efficiency, addressing a wide range of needs and challenges.
* **Data-Related Topics:** We examine critical issues related to training data to optimize training and performance.

### Hiring Research Scientists

We are seeking globally competitive research scientists committed to advancing generational research. As a valued team member, you will have the opportunity to lead your own projects, select your research topics, and collaborate with other members and our network of affiliated centers. We place a high priority on the proactivity and initiative of our new members, viewing these qualities as essential to driving our environment forward. Ideal candidates should possess excellent communication skills and a strong desire to innovate and collaborate, contributing to our dynamic and forward-thinking team.



**Requirements**

* **Publication record**: Proven track record of publications in multiple top-tier conferences and journals in AI/ML (prerequisite).
* **Communication skills**: Exceptional ability to communicate effectively and engage in open discussions.
* **Analytical abilities**: Deep insight and intuition, complemented by outstanding problem-solving abilities.
* **Academic qualification**: Ph.D. (or expected within six months) in Computer Science, Electrical Engineering, or equivalent research experience in these fields.
* **Research experience**: Significant experience in research collaboration and academic writing in relevant domains.

### Hiring Research Interns &#x20;

We are seeking internship members who are passionate about the research topics we focus on. While alignment with our mentioned topics is not mandatory, preference will be given to candidates who are actively studying closely related areas. A key prerequisite for the internship is excellent communication skills, which are necessary for articulating research progress and findings clearly and effectively. This is a full-time, in-person internship role.

**Requirements**

* **Communication skills:** Strong communication skills and openness to constructive discussions and feedback (prerequisite for those who would like to be an internship member).
* **Educational background:** Currently pursuing a PhD or equivalent in Computer Science (CS), Electrical Engineering (EE), Mathematics, or other relevant fields (not mandatory but preferred).
* **Research contributions**: At least one paper was authored by the first author of an AI/ML-related conference.
* **Collaboration experience:** Proven experience in research collaborations and academic writing in related fields.

#### How to Apply

* Please submit your application via this platform to register for our Talent Pool (available in [Korean](https://recruit.navercloudcorp.com/rcrt/view.do?annoId=30001890\&lang=ko)/[English](https://recruit.navercloudcorp.com/rcrt/view.do?annoId=30001891\&lang=en)), where sign-in is required.
  * Application category: Tech > Common > Common > AI (Intern)
  * Fall/Winter Internship may start on September 2nd.

### Recent selected publications (from 2023 to 2024)

Below is a list of recent publications that reflect our areas of interest and ongoing projects. NAVER AI Lab members are distinguished by being displayed in bold text. The corresponding authors and those who have made equal contributions are denoted by \*, and the internship members or visiting researchers who worked in NAVER AI Lab are donated by ✝︎.\


**2024**

* [<mark style="color:blue;">Rotary Position Embedding for Vision Transformer</mark>](https://arxiv.org/abs/2403.13298)**, Byeongho Heo, Song Park, Dongyoon Han, Sangdoo Yun**, ECCV 2024.
* [<mark style="color:blue;">Learning with Unmasked Tokens Drives Stronger Vision Learners</mark>](https://arxiv.org/abs/2310.13593), **Taekyung Kim\*, Sanghyuk Chun, Byeongho Heo, Dongyoon Han\***, ECCV 2024.
* [<mark style="color:blue;">DenseNets Reloaded: Paradigm Shift Beyond ResNets and ViTs</mark>](https://arxiv.org/abs/2403.19588), Donghyun Kim\*, **Byeongho Heo, Dongyoon Han**\*, ECCV 2024.
* [<mark style="color:blue;">SeiT++: Masked Token Modeling Improves Storage-efficient Training</mark>](https://arxiv.org/abs/2312.10105), Minhyun Lee✝︎\*, **Song Park\***, **Byeongho Heo, Dongyoon Han**, Hyunjung Shim, ECCV 2024.
* [<mark style="color:blue;">Leveraging Temporal Contextualization for Video Action Recognition</mark>](https://arxiv.org/abs/2404.09490), Minji Kim✝, **Dongyoon Han, Taekyung Kim\***, Bohyung Han\*, ECCV 2024.
* [<mark style="color:blue;">Model Stock: All we need is just a few fine-tuned models</mark>](https://arxiv.org/abs/2403.19522), Dong-Hwan Jang✝︎, **Sangdoo Yun**\*, **Dongyoon Han**\*, ECCV 2024 (oral presentation).
* [<mark style="color:blue;">HYPE: Hyperbolic Entailment Filtering for Underspecified Images and Texts</mark>](https://arxiv.org/abs/2404.17507), **Wonjae Kim, Sanghyuk Chun, Taekyung Kim, Dongyoon Han, Sangdoo Yun**, ECCV 2024 (oral presentation).
* [<mark style="color:blue;">Similarity of Neural Architectures using Adversarial Attack Transferability</mark>](https://arxiv.org/abs/2210.11407), Jaehui Hwang✝&#xFE0E;**, Dongyoon Han, Byeongho Heo, Song Park, Sanghyuk Chun,** Jong-Seok Lee, ECCV 2024.&#x20;

**2023**

* [<mark style="color:blue;">Neglected Free Lunch - Learning Image Classifiers Using Annotation Byproducts</mark>](https://arxiv.org/abs/2303.17595), **Dongyoon Han**\*, Junsuk Choe\*, Seonghyeok Chun, John Joon Young Chung, Minsuk Chang, **Sangdoo Yun**, Jean Y. Song, Seong Joon Oh, ICCV 2023.
* [<mark style="color:blue;">Switching Temporary Teachers for Semi-Supervised Semantic Segmentation</mark>](https://arxiv.org/abs/2310.18640), Jaemin Na✝︎, **Jung-Woo Ha**, Hyung Jin Chang, **Dongyoon Han**\*, Wonjun Hwang\*, NeurIPS 2023.&#x20;
* [<mark style="color:blue;">Gramian Attention Heads are Strong yet Efficient Vision Learners</mark>](https://arxiv.org/abs/2310.16483), Jongbin Ryu\*, **Dongyoon Han\***, Jongwoo Lim, ICCV 2023.
* [<mark style="color:blue;">Scratching Visual Transformer's Back with Uniform Attention</mark>](https://arxiv.org/abs/2210.08457), Nam Hyeon-Woo✝︎, Kim Yu-Ji, **Byeongho Heo, Dongyoon Han**, Seong Joon Oh, Tae-Hyun Oh, ICCV 2023.&#x20;
* [<mark style="color:blue;">Generating Instance-level Prompts for Rehearsal-free Continual Learning</mark>](https://openaccess.thecvf.com/content/ICCV2023/html/Jung_Generating_Instance-level_Prompts_for_Rehearsal-free_Continual_Learning_ICCV_2023_paper.html), Dahuin Jung✝︎, **Dongyoon Han**, Jiwhan Bang, Hwanjun Song, ICCV 2023 (oral presentation).
* [<mark style="color:blue;">SeiT: Storage-Efficient Vision Training with Tokens Using 1% of Pixel Storage</mark>](https://arxiv.org/abs/2303.11114)**, Song Park\*, Sanghyuk Chun\*, Byeongho Heo, Wonjae Kim, Sangdoo Yun**, ICCV 2023.&#x20;
* [<mark style="color:blue;">The Devil is in the Points: Weakly Semi-Supervised Instance Segmentation via Point-Guided Mask Representation</mark>](https://arxiv.org/abs/2303.15062), Beomyoung Kim, Joonhyun Jeong, **Dongyoon Han**, Sung Ju Hwang, CVPR 2023.
* [<mark style="color:blue;">What Do Self-Supervised Vision Transformers Learn?</mark>](https://openreview.net/forum?id=azCKuYyS74), Namuk Park✝︎, **Wonjae Kim, Byeongho Heo, Taekyung Kim, Sangdoo Yun**, ICLR 2023.

\


