**** | English

[(HuaTuo)]:

BenTsao (original name: HuaTuo): Instruction-tuning Large Language Models With Chinese Medical Knowledge

/(Instruction-tuning) LLaMAAlpaca-ChineseBloom

ChatGPT API

News

[2023/09/24]

[2023/09/12]arxiv

[2023/09/08]arxiv

[2023/08/07]

[2023/08/05] CCL 2023 Demo TrackPoster

[2023/08/03] SCIR

[2023/07/19] Bloom

[2023/05/12] """"

[2023/04/28] Alpaca

[2023/04/24] LLaMA

[2023/03/31] LLaMA

A Quick Start

python3.9+

pip install -r requirements.txt

LoRA

• 1.0Bloom-7B
• Bloom-7B
• Alpaca-Chinese-7BLLaMA
• LLaMA-7B

LoRA

LoRAHugging Face

1. LoRA

2. BloomLoRA

• Hugging Face

3. AlpacaLoRA

• Hugging Face
• Hugging Face

4. LLaMALoRA

• Hugging Face
• Hugging Face

LoRA

**lora-folder-name**/
 - adapter_config.json  # LoRA
 - adapter_model.bin  # LoRA

ChatGLM: ChatGLM-6B-Med

Infer

./data/infer.json

infer

#
bash ./scripts/infer.sh

#
#
bash ./scripts/infer-literature-single.sh

#
bash ./scripts/infer-literature-multi.sh

infer.shbase_modelloralora_weightsinstruct_dir

python infer.py \
	    --base_model 'BASE_MODEL_PATH' \
	    --lora_weights 'LORA_WEIGHTS_PATH' \
	    --use_lora True \
	    --instruct_dir 'INFER_DATA_PATH' \
	    --prompt_template 'TEMPLATE_PATH'

__

./scripts/test.sh

cMeKG

:

{"": "", "": ["", ""], "": ["", "", ""], "": ["", ""], "": [""]}

GPT3.5Prompt

""

: "

2023GPT3.5./data_literature/liver_cancer.json1k``

""16

Finetune

./data/llama_data.json

finetune

bash ./scripts/finetune.sh

LLaMAA100-SXM-80GB102h17mbatch_size=12840G3090/4090(24GB)batch_size

wandb:

https://wandb.ai/thinksoso/llama_med/runs/a5wgcnzt/overview?workspace=user-thinksoso

20233

1. Q: ""

   A: SCIR""

2. Q: """"

   A: LLaMAAlpaca

3. Q:

   A:

4. Q:

   A: LLaMAAlpacabloom-based-based

5. Q: /

   A: requirementscudalorallama-basedllamaloraissue

6. Q:

   A:

• : HIT-SCIR/huozi
• Facebook LLaMA: facebookresearch/llama
• Stanford Alpaca: tatsu-lab/stanford_alpaca
• alpaca-lora by @tloen: tloen/alpaca-lora
• CMeKG king-yyf/CMeKG_tools
• https://yiyan.baidu.com/welcome logo

Citation

: Huatuo: Tuning llama model with chinese medical knowledge

@misc{wang2023huatuo,
   title={HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge},
   author={Haochun Wang and Chi Liu and Nuwa Xi and Zewen Qiang and Sendong Zhao and Bing Qin and Ting Liu},
   year={2023},
   eprint={2304.06975},
   archivePrefix={arXiv},
   primaryClass={cs.CL}
}

Knowledge-tuning Large Language Models with Structured Medical Knowledge Bases for Reliable Response Generation in Chinese

@misc{wang2023knowledgetuning,
      title={Knowledge-tuning Large Language Models with Structured Medical Knowledge Bases for Reliable Response Generation in Chinese}, 
      author={Haochun Wang and Sendong Zhao and Zewen Qiang and Zijian Li and Nuwa Xi and Yanrui Du and MuZhen Cai and Haoqiang Guo and Yuhan Chen and Haoming Xu and Bing Qin and Ting Liu},
      year={2023},
      eprint={2309.04175},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

The CALLA Dataset: Probing LLMs Interactive Knowledge Acquisition from Chinese Medical Literature

@misc{du2023calla,
      title={The CALLA Dataset: Probing LLMs' Interactive Knowledge Acquisition from Chinese Medical Literature}, 
      author={Yanrui Du and Sendong Zhao and Muzhen Cai and Jianyu Chen and Haochun Wang and Yuhan Chen and Haoqiang Guo and Bing Qin},
      year={2023},
      eprint={2309.04198},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}