Delphi Divide a TStringList Contents Evenly into an array

Hey Guys,

I am attempting to divide a tstringlist of accounts evenly among 10 threads.

I am inserting the "batches" of accounts into different arrays.

So if i had 30 accounts, i would be dividing the list up into batches of 3 accounts and inserting them into their appropriate account array.. that way each thread can process 3 accounts and speed up things

problem I am running into.. is when the division is not even.. I've tried 3 different variations of code but when i change up the amount of accounts i get all funky results. I am hoping one of you have a very clean and simple way to do this without any errors.. regardless of how many accounts i use.

Remember i am only using 10 threads.. so essentially the list has to be split up between 10 threads evenly.
nickdelphi777Asked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

nickdelphi777Author Commented:
Please keep in mind.. i want it to always use 10 threads unless theres less than 10 accounts of course.

So if I have 15 accounts.. i do not want it to divide up by 2. and only use 7 threads.. I would like it to put say 1 account in most.. then 2 here.. and 2 there.

please help i have been stuck with this for hours
0
ste5anSenior DeveloperCommented:
Just round-robin them.
0
nickdelphi777Author Commented:
Thats not really what I'm after...
0
Get your problem seen by more experts

Be seen. Boost your question’s priority for more expert views and faster solutions

nickdelphi777Author Commented:
Nevermind.. it popped in my head.. just loop through entire list and keep adding to the threads.. 1 to 10, 1 to 10:

imageamount := memo1.lines.count;
threads :=10;
icount := 0;
for i := 0 to imageamount-1 do begin
  memo2.lines.add('Insert '+memo1.lines[i]+ ' into '+inttostr(icount));
 if(icount = threads-1) then begin
  icount := 0;
 end else begin
  inc(icount);
 end;
end;

Open in new window

0
jimyXCommented:
Compact code:
  imageamount := memo1.lines.count;
  threads :=10;
  for i := 0 to imageamount-1 do begin
    memo2.lines.add('Insert '+memo1.lines[i]+ ' into '+inttostr(i mod threads));
  end;

Open in new window

0
jimyXCommented:
In case you want to know beforehand how many to cut from accounts, in one step to assign to the processing threads in same order, then:
function ImagesToProcess(thrdCount, thrdindx, imgs:Integer):integer;
begin
  Result:= 0;

  if thrdindx <= ((imgs) mod thrdCount) then
    begin
     Result:= ((imgs-1) div thrdCount)+1;
    end
  else
    begin
      Result:= ((imgs-1) div thrdCount);
    end;
end;

procedure TForm1.Button1Click(Sender: TObject);
var
  thrd, threads, imageamount:integer;
begin
  imageamount:= memo1.lines.count;
  threads:= 10;

  for thrd:= 1 to threads do
    begin
      memo2.lines.add('thread '+ inttostr(thrd)+' will get '+ IntTostr(ImagesToProcess(threads, thrd, imageamount)));
    end;
end;

Open in new window


If you want to see that working as you tested in your last comment then:
procedure TForm1.Button2Click(Sender: TObject);
var
  k, thrd, threads, img, imageamount:integer;
  str:String;
begin
  Memo2.Clear;
  imageamount := memo1.lines.count;
  threads :=10;
  img:= 0;

  for thrd:= 1 to threads do
    begin
      str:= '';

      if thrd <= ((imageamount) mod threads) then begin
        for k:= 0 to ((imageamount-1) div threads) do
          begin
            str:= str+ memo1.lines[img];
            inc(img);
          end;
      end
      else begin
        for k:= 0 to ((imageamount-1) div threads)-1 do
          begin
            str:= str+ memo1.lines[img];
            inc(img);
          end;
      end;

      memo2.lines.add('thread '+ inttostr(thrd)+' will have '+ Str);
    end;
end;

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Geert GOracle dbaCommented:
you don't even need that, just use Otl

use a pipeline from omnithreadlibrary ...
http://otl.17slon.com/index.htm

you only need 1 stage, this pipeline can buffer up to 10000 elements
optionally add an message proc
Numtasks indicates 10 threads

 
 fPipeLine :=
    Parallel.PipeLine
    .Throttle(10000)
    .Stage(
      TaskProcessItem, Parallel.TaskConfig.OnMessage(MsgProcessItem)
      ).NumTasks(10)
  .Run;

Open in new window


Loading the queue:
for I := 0 to Strings.Count-1
  fPipeLine.AddInput(Strings[I]);

Open in new window


the thread processing code
procedure TaskProcessItem(const input, output: IOmniBlockingCollection; const task: IOmniTask);
  Value: TOmniValue;
begin
  while not Task.Terminated do
  begin
    if input.TryTake(Value, 1000) then
    begin
        // Thread processing code: 
        // string for this is : Value.AsString

        // optionally send a message
        task.Comm.Send(0, 'Processed item : ' + Value.AsString);
    end;
  end;
end;

Open in new window


procedure MsgProcessItem(const task: IOmniTaskControl; const msg: TOmniMessage);
begin
   fNotifyEvent(Self, msg.MsgData.AsString);
end;

Open in new window

0
Geert GOracle dbaCommented:
the 10 threads will take the next item when they have processed the previous
if no items are in the queue, they wait for 1 second and then retry

the next thing about this pipeline technique is that you don't need to divide anything yourself

the numtasks (15) would setup 15 threads.
numtasks(2) would only setup 2 threads
0
ste5anSenior DeveloperCommented:
@nickdelphi777: Your code IS the round-robin algorithm.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Delphi

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.