#pre-training Repositories
Discover 18 active open source GitHub repositories tagged with #pre-training, accumulating 43,729 total stars (averaging 2,429 stars per project). Leading this topic is RUCAIBox/LLMSurvey with 12,165 stars.
18 repositories total
The official GitHub page for the survey paper "A Survey of Large Language Models".
Data processing for and with foundation models! ๐ ๐ ๐ฝ โก๏ธ โก๏ธ๐ธ ๐น ๐ท
Awesome resources for in-context learning and prompt engineering: Mastery of the LLMs such as ChatGPT, GPT-3, and FlanT5, with up-to-date and cutting-edge updates.
Papers about pretraining and self-supervised learning on Graph Neural Networks (GNNs).
Code for TKDE paper "Self-supervised learning on graphs: Contrastive, generative, or predictive"
Awesome list for research on CLIP (Contrastive Language-Image Pre-Training).
A professional list on Large (Language) Models and Foundation Models (LLM, LM, FM) for Time Series, Spatiotemporal, and Event Data.
The repository of ET-BERT, a network traffic classification model on encrypted traffic. The work has been accepted as The Web Conference (WWW) 2022 accepted paper.
[ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
[NeurIPS 2020] "Graph Contrastive Learning with Augmentations" by Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen
Frequently Asked Questions
What is the most popular #pre-training repository on GitHub?
The top ranked open source #pre-training project on GitHub is RUCAIBox/LLMSurvey with 12,165 stars. The official GitHub page for the survey paper "A Survey of Large Language Models".
Which programming languages are most used in #pre-training repositories?
Among the top ranked #pre-training projects on GitHub, the most common programming languages are Python, Jupyter Notebook.
How is the #pre-training repository ranking calculated?
We track 18 active open source repositories tagged with #pre-training (500+ stars), totaling 43,729 stars with an average of 2,429 stars per repository. Rankings update daily from the GitHub API.