Listview and Scrollbox with embeded Editboxes Scroll problem

I have built a listview inside of a scrollbox and made it so that when you select a row it pulls up a panel with Editboxes in it that automaticaly size to the lisview row and alow you to edit the data in the row.

 Here is some of the code I am using to do this.

//Set Listbox To full Lenth
procedure TForm1.SetListViewSize(aListView:TListView);
var
 i, W, H: integer;
begin
  with aListView.Items[0].DisplayRect(drBounds)
    do H:=((Bottom-Top)+1) * aListView.Items.Count;

  W:=0;

  for i:=0 to aListView.Columns.Count-1
    do
   inc(W, aListView.Column[i].Width + 3);
   aListView.SetBounds(0, 0, W, H);
end;




//Load Listview
procedure TForm1.Button1Click(Sender: TObject);
begin
    ADOMain.SQL.Text := 'Select * From Items Where type = ''Labor''  order by Itemname';
    ADOMain.Open;

    ListView1.Items.Clear;
    while not ADOMain.Eof do begin
      NewItem := ListView1.Items.Add;
      NewItem.Data := Pointer(ADOMain.FieldByName('ItemID').AsInteger);
      NewItem.Caption := ADOMain.FieldByName('ItemName').AsString;
      NewItem.SubItems.Add(ADOMain.FieldByName('ItemDescription').AsString);
      ADOMain.Next;
    end;
    ADOMain.Close;
    SetListViewSize(ListView1);
end;




//Set Panel and editboxes into listview
procedure TForm1.ListView1MouseDown(Sender: TObject; Button: TMouseButton;
  Shift: TShiftState; X, Y: Integer);
var
  ColNum, nWdt: integer;
  R: TRect;
  LVI:TListItem;
begin

  PANEL1.Visible := True;

 with ListView1 do
   begin
    ColNum:=0;
    nWdt:=0;

  LVI:=GetItemAt(X, Y);

  if LVI <> nil then begin
      R:=LVI.DisplayRect(drLabel);
      Panel1.SetBounds(nWdt+Left+3,R.Top+Top+3,450, R.Bottom-R.Top);
  end;

  Panel1.BringToFront;

  if Assigned(ListView1.Selected) then begin
    Edit1.text := ListView1.Selected.Caption;
    Edit2.Text := ListView1.Selected.SubItems[0];
  end;

 end;

end;

When you scroll the scrollbox it keeps the panel and editboxes synonomus with the listview row you selected. All of this works great, however if you have a long list and you scroll the scrollbox down then select a row it automatically scrolls the scrollbox to where the selected row is the last row in the scrollbox. I need the scrollbox to not change but leave the selected row exactally where it was selected.