Artificial intelligence (AI) has come a long way in just the last year, and one area that has seen significant progress is natural language processing. This technology allows computers to understand and generate human-like language, enabling them to communicate with people in a more natural and intuitive way. With new technology comes new CMS-like software to enable anyone to participate. Enter MagickML, a multishot system builder that leverages the capabilities of GPT-3 and other OpenAI language models to create AI assistants and characters with complex communication patterns. It is, in a sense, a WordPress of AI in that it manages AI themes and content. In this post, we'll take a closer look at Magick and explore its potential to revolutionize the way we interact with AI in virtual worlds, particularly in 3OV.
What is Magick?
MagickML is a JavaScript-based framework that allows game designers and developers to rapidly create powerful natural language systems and prototype NPC logic in any web based experience. It utilizes a visual coding style interface, making it easy for users to create "spells" that define complex logic and queries to generate responses. These spells are almost like WordPress themes for AI, allowing users to customize the behavior and communication patterns of their AI assistants and characters. Being JavaScript driven allows for a much wider range of developer to participate. And the icing on the cake, it is open source and Apache Licensed!
One of the key features of Magick is its ability to integrate with GPT-3 and any other OpenAI language models to logically respond to requests. This enables users to leverage the advanced capabilities of these models to create AI assistants and characters that can understand and generate human-like language. You can use any mixture of language models to generate a final response.
The framework also allows users to incorporate memory recall or factual biographical information into their responses, as well as any other logic they would like to include to make it feel more natural. Between the final output of an entity, users are able to utilize JavaScript logic to parse many responses from OpenAI to build a final query that generates a more human-like response. Where this differs from something like chatGPT is its ability to send multiple request before giving the end user final output. This gives you room to tweak all the funny bits of language models where a single query might not give you the response you want.
The Potential of Magick:
The potential of Magick facilitating the creation of complex communication patterns with OpenAI language models is vast. In the gaming industry, Magick could be used to create more immersive and engaging experiences for players using AI characters with personality traits, making the game world feel more real and alive.
I found the potential of chatbots and virtual assistants particularly interesting for 3D web applications. By using Magick to create AI assistants, businesses and organizations could provide more personalized and efficient customer service. Your AI assistant can sell for you within your website. And with 3OV this could even be in a 3D world!
I built a Magick driven proof of concept in 3OV that allows WordPress site owners to define a URL where their Magick instance lives and make post requests from in-world to display responses above an avatar's head. This avatar is given a personality through "spells" and is able to remember facts about each person that talks thanks to the awesome work Magick contributor m00n has built and shared with their Eliza entity.
Example Integration
The following integration of Magick in 3OV very basically shows how you can integrate spells and responses in your apps.
function ChatBox(props) {
const handleChange = async (event) => {
event.preventDefault();
};
const handleSubmit = async (event) => {
event.preventDefault();
// Get the value of the input element
const input = event.target.elements.message;
const value = input.value;
// Send the message to the localhost endpoint
const client = 1;
const channelId = "three";
const entity = 11;
const speaker = "antpb";
const agent = "aiko";
const channel = "homepage";
try {
const spell_handler = "four";
const spell_version = "latest";
const url = encodeURI( `https://localhost:8001/spells/${spell_handler}/${spell_version}` )
const response = await axios.post(`${url}`, {
inputs: {
Input: value,
Speaker: speaker,
Agent: agent,
Client: client,
ChannelID: channelId,
Entity: entity,
Channel: channel,
},
}).then((response) => {
const data = response.data;
const outputs = data.outputs;
const outputKey = Object.keys(outputs)[0];
const output = outputs[outputKey];
props.setMessages([...props.messages, output]);
});
} catch (error) {
console.error(error);
}
};
return (
<div style={{ marginTop: "-140px", position: "relative", bottom: "15%", left: "5%", width: "50%", height: "10%" }}>
<form style={{display: "flex"}} onSubmit={handleSubmit}>
<input type="text" name="message" onInput={handleChange} onChange={handleChange} />
<button type="submit">Send</button>
</form>
</div>
);
}Code language: JavaScript (javascript)In this example implementation of Magick in the 3OV plugin, the ChatBox function handles the submission of user input and sends it to a localhost endpoint through an HTTP post request. (In this case I have Magick running locally but you could install this on any server.) The request includes a few parameters like the input value, speaker and agent names, client and channel IDs, and entity and channel values. These parameters are passed to the endpoint as an object in the request body and is used to process and store information about the context of the conversation in Magick.
Upon submission, the function sends the message to a specified Magick spell handler and version, which are defined in the URL of the request. The spell handler and version are set to "four" and "latest" respectively in my example. You could have many spells and route chats to different spell handlers to dynamically make the requests more cost efficient or more technical depending on your needs.
const spell_handler = "four";
const spell_version = "latest";
const url = encodeURI( `https://localhost:8001/spells/${spell_handler}/${spell_version}` )
const response = await axios.post(`${url}`, {
inputs: {
Input: value,
Speaker: speaker,
Agent: agent,
Client: client,
ChannelID: channelId,
Entity: entity,
Channel: channel,
},
}).then((response) => {
const data = response.data;
const outputs = data.outputs;
const outputKey = Object.keys(outputs)[0];
const output = outputs[outputKey];
props.setMessages([...props.messages, output]);
});
} catch (error) {
console.error(error);
}
Code language: JavaScript (javascript)Upon receiving the request, Magick processes the input and other parameters using the specified spell, and returns a response in the form of an object containing output data. The ChatBox function retrieves the output data and adds it to the list of messages using setMessages. This state is passed to a lower component where the AI assistant avatar file is handled. The message populates above the avatars head and closes the loop of conversation!
You may notice in the screenshot above there’s json being output with a tone property. That is GPT-3 deciding the tone of the message it is sending! You could easily map this to VRM avatar facial expressions so the AI can naturally interact with you in-world. I plan to get more granular and give the spell control of the default VRM emotion parameters.
Voice recognition and responses are 100% possible today thanks to the work of Magick contributors. Overall, this implementation of Magick within the 3OV plugin demonstrates simply how the framework can be used to create interactive and personalized communication patterns with AI in a 3D environment. This can go way further and is only limited by imagination.
Pricing and Caution
It's important to note that OpenAI queries could potentially be expensive to use, depending on how many shots to the AI you need to make in a spell. For example, if you are using a spell that costs around $0.02 per query and you spell has multiple queries, this could add up quickly if you are making frequent requests. It's important to be mindful of this when using Magick and to consider whether it is cost-effective for your needs. Adjusting language models to cheaper options is another way to ease costs. In my testing I have short circuited the AI bits of a spell to return static responses to mock against as I develop ideas.
I would encourage using Magick as a local personal AI assistant or implementing safeguards in your Magick spells to recognize when someone is abusing the system. This could include defaulting to basic responses rather than querying GPT when abuse is detected. These are very early days for this project so keep your instances locked down as auth and other gated features work their way into the framework.
Conclusion:
Magick is a powerful tool for creating AI assistants and characters with complex communication patterns. Its ability to integrate with GPT-3 and other OpenAI language models allows users to leverage the advanced capabilities of these models to create sophisticated responses. The potential applications of Magick are numerous, and it could revolutionize the way we interact with AI in a wide range of industries, from gaming to customer service. I can’t say for certain how we will officially launch integrations for this in 3OV, but the important part is that it is now possible and we will find the right way to launch support through a NPC block. I’m excited to continue contributing to this framework as it takes shape!
