NameValueCollection

Depending on how the data is arranged, you might be able to use an ILookup, which acts like the Dictionary(Of String, List(Of String))  mentioned by kaufmed.

Module Module1
   Sub Main()
      Dim lst_str = New List(Of String) From
      {
            "A,1,", "A,2", "A,3", "A,4",
            "B,5", "B,6", "B,7", "B,8",
            "C,9", "C,10,", "C,11,", "C,12"
      }

      Dim lkup_s2s As ILookup(Of String, String) =
      (
         From str In lst_str
         Let arr = str.Split(",")
         Select New With {.key = arr(0), .value = arr(1)}
      ).ToLookup(Function(k) k.key, Function(v) v.value)
   End Sub
End Module

Select allOpen in new window

If the values were stored as numbers would it make thing easier. I thought that a Namedvaluecollection was used when you have a single key and multiple values.

Well, there is a KeyValuePair(Of TKey, TValue),  that will store one of those groupings.

Dim kvp As New KeyValuePair(Of String, List(Of Integer))

Select allOpen in new window

will store the data as you expect for ("A": 1,2,3,4), for instance.
You would need another one for B and another one for C as you can only have one KEY in that object.

If you wanted them all in the same collection, you would need either a Dictionary(Of String, List(Of Integer))  or the ILookup(Of String, Integer).
With the latter two, you can have multiple KEYS in the collection.

I want to store a key which will always be a unique string and four double values associated with each key. What would you suggest I use and how would I add and retrieve the values?

If the values were stored as numbers would it make thing easier.

It's not so much the type of the data; rather it's the type of collection you are using.

I want to store a key which will always be a unique string and four double values associated with each key.

May I inquire as to how you are using them? I still maintain my original suggestion, but perhaps I don't understand your requirement.