uploaded readme

README.md

@@ -0,0 +1,95 @@
+Quantization made by Richard Erkhov.
+
+[Github](https://github.com/RichardErkhov)
+
+[Discord](https://discord.gg/pvy7H8DZMG)
+
+[Request more models](https://github.com/RichardErkhov/quant_request)
+
+
+nova-1.3b-bcr - bnb 8bits
+- Model creator: https://huggingface.co/lt-asset/
+- Original model: https://huggingface.co/lt-asset/nova-1.3b-bcr/
+
+
+
+
+Original model description:
+---
+license: bsd-3-clause-clear
+---
+# Nova: Generative Language Model For Assembly Code
+
+## Abstract
+Binary code analysis is the foundation of crucial tasks in the security domain; thus building effective binary analysis techniques is more important than ever. Large language models (LLMs) although have brought impressive improvement to source code tasks, do not directly generalize to assembly code due to the unique challenges of assembly: (1) the low information density of assembly and (2) the diverse optimizations in assembly code. To overcome these challenges, this work proposes a hierarchical attention mechanism that builds attention summaries to capture the semantics more effectively and designs contrastive learning objectives to train LLMs to learn assembly optimization. Equipped with these techniques, this work develops Nova, a generative LLM for assembly code. Nova outperforms existing techniques on binary code decompilation by up to 14.84 -- 21.58% higher Pass@1 and Pass@10, and outperforms the latest binary code similarity detection techniques by up to 6.17% Recall@1, showing promising abilities on both assembly generation and understanding tasks.
+
+## Introduction of Nova
+Nova is pre-trained with the language modeling objective starting from DeepSeek-Coder checkpoints, using the disassembly code from [AnghaBench](https://github.com/albertan017/LLM4Decompile) and C/C++ program compiled from [The-Stack](https://huggingface.co/datasets/bigcode/the-stack).
+
+This is the repository of the instruciton-tuned model of Nova that is good at binary code recovery, with 1.3B parameters.
+The other models in this series:
+- [Nova-1.3b](https://huggingface.co/lt-asset/nova-1.3b): Foundation model for binary code with 1.3B parameters.
+- [Nova-6.7b](https://huggingface.co/lt-asset/nova-6.7b): Foundation model for binary code with 6.7B parameters.
+- [Nova-6.7b-bcr](https://huggingface.co/lt-asset/nova-6.7b-bcr): Nova-6.7b model further instruction-tuned for binary code recovery.
+
+## Usage
+```python
+from transformers import AutoTokenizer
+from modeling_nova import NovaTokenizer, NovaForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained('lt-asset/nova-1.3b-bcr', trust_remote_code=True)
+if not torch.distributed.is_initialized() or torch.distributed.get_rank() == 0:
+    print('Vocabulary:', len(tokenizer.get_vocab()))    # 32280
+tokenizer.pad_token = tokenizer.eos_token
+tokenizer.pad_token_id = tokenizer.eos_token_id
+nova_tokenizer = NovaTokenizer(tokenizer)
+
+model = NovaForCausalLM.from_pretrained('lt-asset/nova-1.3b-bcr', torch_dtype=torch.bfloat16).eval()
+
+# load the humaneval-decompile dataset
+data = json.load(open('humaneval_decompile_nova_1.3b.json', 'r'))
+for item in data:
+    print(item['task_id'], item['type'])
+
+    prompt_before = f'# This is the assembly code with {item["type"]} optimization:\n<func0>:'
+    asm = item['normalized_asm'].strip()
+    assert asm.startswith('<func0>:')
+    asm = asm[len('<func0>:'): ]
+    prompt_after = '\nWhat is the source code?\n'
+    
+    inputs = prompt_before + asm + prompt_after
+    # 0 for non-assembly code characters and 1 for assembly characters, required by nova tokenizer
+    char_types = '0' * len(prompt_before) + '1' * len(asm) + '0' * len(prompt_after)
+    
+    tokenizer_output = nova_tokenizer.encode(inputs, '', char_types)
+    input_ids = torch.LongTensor(tokenizer_output['input_ids'].tolist()).unsqueeze(0)
+    nova_attention_mask = torch.LongTensor(tokenizer_output['nova_attention_mask']).unsqueeze(0)
+
+    outputs = model.generate(
+        inputs=input_ids.cuda(), max_new_tokens=512, temperature=0.2, top_p=0.95,
+        num_return_sequences=20, do_sample=True, nova_attention_mask=nova_attention_mask.cuda(),
+        no_mask_idx=torch.LongTensor([tokenizer_output['no_mask_idx']]).cuda(), 
+        pad_token_id=tokenizer.pad_token_id, eos_token_id=tokenizer.eos_token_id
+    )
+    item['infer_c_func'] = []
+    for output in outputs:
+        item['infer_c_func'].append({
+            'c_func': tokenizer.decode(output[input_ids.size(1): ], skip_special_tokens=True, clean_up_tokenization_spaces=True)
+        })
+
+    json.dump(data, open(f'humaneval_decompile_nova_1.3b.json', 'w'), indent=2)
+```
+
+## Citation
+```
+@misc{jiang2024nova,
+      title={Nova: Generative Language Models for Assembly Code with Hierarchical Attention and Contrastive Learning}, 
+      author={Nan Jiang and Chengxiao Wang and Kevin Liu and Xiangzhe Xu and Lin Tan and Xiangyu Zhang},
+      year={2024},
+      eprint={2311.13721},
+      archivePrefix={arXiv},
+      primaryClass={cs.SE},
+      url={https://arxiv.org/abs/2311.13721}, 
+}
+```
+