| 일 | 월 | 화 | 수 | 목 | 금 | 토 |
|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | |||
| 5 | 6 | 7 | 8 | 9 | 10 | 11 |
| 12 | 13 | 14 | 15 | 16 | 17 | 18 |
| 19 | 20 | 21 | 22 | 23 | 24 | 25 |
| 26 | 27 | 28 | 29 | 30 |
Tags
- docker
- Linux
- Python
- format
- Database
- Windows
- numpy
- AI
- LLM
- Converting
- file
- Container
- Laravel
- TORCH
- pytorch
- ubuntu
- GitLab
- Mac
- list
- KAKAO
- PostgreSQL
- pandas
- Package
- DB
- Flask
- judge
- evaluation
- Paper
- 책갈피
- git
Archives
- Today
- Total
목록embedding (1)
Daily Develope
[AI] jinaai Embedding 모델 샘플코드
모델 로딩 (huggingface)import torchfrom transformers import AutoModel, AutoTokenizermodel_name = "jinaai/jina-embeddings-v3"tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModel.from_pretrained(model_name)Tokenizer 테스트# Token 출력 (Dict. id값)embedding = tokenizer("한 남자가 밧줄을 타고 올라가고 있다.")print(len(embedding[0])) # 14# [0, 3103, 64757, 713, 6, 246028, 60354, 413, 156860, 87179, 10843..
AI
2024. 12. 3. 16:09