IMAGINE-7B-Instruct

Interaction Model for Advanced Graphics Inference and Exploration

This Large Language Model (LLM) is a fine-tuned version of Mistral-7B-Instruct-v0.1. It is designed to integrate the conversational method into the process of generating image prompts. This model excels in understanding and responding to prompts related to image generation through an interactive dialogue. This innovative approach allows users to engage in dialogues, providing textual prompts that guide the model in generating corresponding sets of tokens. These tokens, in turn, serve as dynamic prompts for subsequent interactions.

The main purpose of IMAGINE was to better understand how fine-tuning an LLM works,

Model Details

Model Architecture

This instruction model is based on Mistral-7B-v0.1, a transformer model with the following architecture choices:

• Grouped-Query Attention
• Sliding-Window Attention
• Byte-fallback BPE tokenizer

Training Details

Data

The dataset used to fine-tune this model has been entirely created by Syntonomous and does not contain any external sources. For more information on how the original Mistral-7B-Instruct-v0.1 was fine-tuned, please refer to their model page.

Compute Infrastructure

• Hardware:
  • 4x Nvidia Tesla V100S 32GiB
  • 160 GiB RAM
  • 52 vCores CPU
• Compute Region: Europe
• Training Effective Duration: 6 hours/resource (=24 hours)
• Carbon Emitted: 0.72kg CO² (300W * 24h = 7.2 kWh * 0.1kg CO²)

✦ Syntonomous
Learn to share — Share to learn