.Net: add amazon nova support (text) only #10021
Conversation
markwallace-microsoft
added
.NET
github-actions
bot
changed the title
|
@microsoft-github-policy-service agree
@microsoft-github-policy-service agree |
| @@ -76,7 +77,7 @@ internal async Task<IReadOnlyList<TextContent>> InvokeBedrockModelAsync( | |||
| try | |||
| { | |||
| var requestBody = this._ioTextService.GetInvokeModelRequestBody(this._modelId, prompt, executionSettings); | |||
| using var requestBodyStream = new MemoryStream(JsonSerializer.SerializeToUtf8Bytes(requestBody)); | |||
| using var requestBodyStream = new MemoryStream(JsonSerializer.SerializeToUtf8Bytes(requestBody, new JsonSerializerOptions { DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull })); | |||
There was a problem hiding this comment.
Recommend using the options serializer as a static instance, avoiding instantiation each request.
|
|
||
| /// <summary> | ||
| /// Converts the streaming JSON into <see cref="IEnumerable{String}"/> for output. | ||
| /// </summary> | ||
| /// <param name="chunk">The payloadPart bytes provided from the streaming response.</param> | ||
| /// <returns><see cref="IEnumerable{String}"/> output strings.</returns> | ||
| internal IEnumerable<string> GetTextStreamOutput(JsonNode chunk); | ||
| internal IEnumerable<string> GetTextStreamOutput(string modelId, JsonNode chunk); |
There was a problem hiding this comment.
With the latest merge from my changes this signature needs to be updated to return StreamingChatMessageContent instead of string.
|
|
||
| var requestBody = new TitanRequest.TitanTextGenerationRequest() | ||
| if (modelId.StartsWith("amazon.nova-", StringComparison.OrdinalIgnoreCase)) |
There was a problem hiding this comment.
Strongly suggest to move this service to AmazonNovaService and not mix this with titan as the models (API contracts) are also different.
When moving to AmazonNovaService implementation, revert the extra modalId from the interface and feel free to rename the existing AmazonService to titan specific AmazonTitanService.
It's clear that the responsibility to know with model should be only in the factory for this perspective, avoiding bringing extra complexity to the service level.
| @@ -75,7 +75,7 @@ public ConverseRequest GetConverseRequest(string modelId, ChatHistory chatHistor | |||
| } | |||
|
|
|||
| /// <inheritdoc/> | |||
| public IEnumerable<string> GetTextStreamOutput(JsonNode chunk) | |||
| public IEnumerable<string> GetTextStreamOutput(string modelId, JsonNode chunk) | |||
There was a problem hiding this comment.
Note
Apply to all services
Revert this change as all the services were designed to not use the modelId information, worth having a specific service to handle the different Nova requests.
| public IEnumerable<string> GetTextStreamOutput(string modelId, JsonNode chunk) | |
| public IEnumerable<string> GetTextStreamOutput(JsonNode chunk) |
| /// Prompt execution settings for Nova Text Generation | ||
| /// </summary> | ||
| [JsonNumberHandling(JsonNumberHandling.AllowReadingFromString)] | ||
| public class AmazonNovaExecutionSettings : PromptExecutionSettings |
There was a problem hiding this comment.
Seeing this specific Nova execution settings feels even stronger the need to have a dedicated concrete service for Nova models within Amazon's folder.
|
@EriksonBahr Thanks for the amazing contribution, some extra conflicts needs to be addressed. Happy new year! 🎉 |
RogerBarreto
added
the
PR: feedback to address
Motivation and Context
Description
Contribution Checklist