Zebra 105sl stops printing intermittently from AS400

Would the sites happen to be connected via a site-to-site VPN?   Or are they MPLS or some other WAN technology?

~ Jon Snyderman

I'm assuming these are using LPR/LPD over TCP:  Look at the printer device description (WRKDEVD) or remote outq description (WRKOUTQD) to verify the connection type.

Next time, while the print job is still stuck in SND status, look at the job logs for the writer associated with the failed print job.  (WRKJOB nameofprintergoeshere) A lot of times there will be an error message that will help with further troubleshooting.

There can be a lot of causes.  

Based on your description, I'm betting you are running out of LPR/LPD ports.  AS/400 only allocates a limited number of ports for outbound LPR connections.  If you have a lot of fairly busy printers, plus a long timeout, you can get this intermittent failed printing issue.  If that's the problem, you'll get an error message in the writer's job log (TCP100D is the message ID).

IBM has a good troubleshooter for remote outq problems:

http://www-01.ibm.com/support/docview.wss?uid=nas12a414d2d7d7ab7e9862568be00722993

- Gary Patterson

We are on MPLS

Thanks Gary I will try looking at it when it fails next and post results

Yes it is TCP

Ok it happened this morning and here is what I see when it gets stuck sending:

OPT        FiLE             USER       USER DATA   STS   PAGES   COPIES  FORM TYPE  PTY
___           QSYSPRT    JOHND    TXTPRT          SND     1            1            *STD             5

There is no WRKDEVD on the output queue but WRKOUTQD offers:
Status:  
   Writer Active:                           Y
   Writer name(s) if active          SHIP2
   Output queue held                  N
Max spooled file size:
   Number of pages:                    *NONE
   Starting time
   Ending time
Writers to autostart:                    1
Display any file                             *NO
Job Seperators                              0
Operator controlled                     *YES
Order fo files in queue                 *FIFO
Data queue:                                   *None
Authority to check                         *Owner
Remote system                              *Interior

Remote printer queue                   portlf1

Queue for writer messages           QSYSLOG
    Library                                              *LIBL
Connection Type                             *IP
Internet Address                             10.20.5.41
Destination type                              *OTHER
Host Print Transform                      *YES
MANUF TYPE and MODEL               *WSCST
Workstation customizing object     Z7XIWSCST
   Libarary                                              TLABARCODE
Image configuration                        *NONE
Destination options                         *NONE

Print seperator page                        *NO
User defined option                         *NONE
User defined object:
   Object                                              *NONE
      Library
User driver program                          *NONE
      Library
Spooled file ASP                                  *SYSTEM
Text Description                                  Zebra printer located near ship2

Queue for messages above said QSYSLOG but doing a DSPMSG QSYSLOG nothing in there for today.

Is there a way to determine the # of ports available for LPR connections and what are normal timeout values.  I have also included some screen shots of the web interface of the printer as we have done a lot of tweaking with the timeout.

Any help would be appreciated.
zebra.docx

Not sure if you recovered it yet, but there should be a job with the name of the printer and a type of WTR.    CHeck that job log for any additional indications.    

Other than Gary's point about the connection limit, in my personal experience, this is almost always a communications or network time-out issue.    

Can you post the config form one of the 105SLs?

~Jon Snyderman

Are you thinking wrkjob <ship2>.  I have nothing in 10 (review job log).  I will try to catch that next tiem.

Where can I check the connection limit.  I attached a .doc with the web config of the printer showing the timeout value.  Is that what config your want or what command can I run?

Hi  bergquistcompany,

We have round 50 Zebra (and stripe) printers in my customers company, and had the same problem for years, IBM and the Zebra supplier investigate it and were not able to solve it. It seems to help a little to set the tim-out value to max, but it didn't solve the problem.

We changed the most important programs to send the label by FTP instead of using a "normal" printer and the problems are solved for these programs.

For FTP you only have to PUT the result to the printer

Regards,
Murph

Exactly Murph, that's why I asked for the config.    Thx.  

BerquestCo, you can check that setting, but the whole config might uncover something.

The FTP idea is a cool one, but then you loose all your printer controll.  You have to weigh the benefits of that solution.

~Jon Snyderman

IBM says it's a network issue, but it seems so many others have it too.
What command would get a complete config?  I will post for you.

@ jsnyderman

Hi JS,

Zebra printers have all print control inside the printer data, the less print control the better it is. Zebra and Stripe printers use the language ZPL or ZPL-II what look like this:

^XA
^FO10,10^A0N,40,40^FDHELLO WORLD^FS
^PQ1
^XZ 

Select allOpen in new window

Where ^xx is the operation code :-)

I have attached this prevriously above.  Maybe you can't see the document I attached.
The configuration is there and the timeout is 300.  Have you found altering this to fix the issue?

Sorry, thats not the config that I am looking for.   Using Murpheys instructions to access the web interface, you can get a full config.  It should be about 1-2 pages long.     Just cut and paste that in.   Yes, you may want to increase the time-out.

~Jon Snyderman

Ok here it is.  Check timeout is set to No.  Did you find enabling and extending this helped?
Capture.JPG

Hi bergquistcompany,

To check if the Time-out is the problem, you cen set it to "Yes" and set the timeout value to 30 seconds, then you probably will have the problem very quick. and can be sure that it was a time-out issue. keep the setting to Yes and set the value to max (I think it will be 999)

If the interval between problems, remain the same, you probably have an other issue.
In that case... When printers stop responding, what do you have to to to get them working again?

If the connection time-out checking is NO, this shouldn't be the issue.   You can raise the timer to 999 but it really should not be relevant.  

There is also a full report in the web interface of all settings.  That may help, but the settings I was after were what you showed.   How about the device or outq description on the 400?  Can you send that?  

Have you been able to identify a period of time after which it fails or any other common situation?  

Did IBM ask you to set up a COMM trace?   Were they able to identify which side was doing the connection dropping?

Lots of questions.  Sorry, but just trying to come up with something concrete to start from.

~Jon Snyderman

Set the timer to 30 and Yes and it hasn't gotten stuck yet so do not think that is the issue.

There is no wrkdevd because it is a remote output q, but the object description is attached.

No it is so random.

Thanks I am too!!!  Any help would be great.
Object-Description.docx

When printers stop responding, what do you have to to to get them working again?

Lets take a look at the outq description.

We have to stop the writer, power off the printer,, power on the printer and start the writer.

can you try the following:

copy the label data to a PC text file e.g. Label.txt.
ONLY the printerdata starting with ^XA ending with ^XZ.

If the error is there just open FTP to the printer and use PUT Label.txt to send the contents of the label.
That way you can test if the printer is accepting data.
If it is'n printing there is a big chance that that is the reason that the spool writer have  problems to reach the printer.

Good test.

bergquistcompany, just to be clear, you are going to run Murpheys test just after it stops working from the 400 before recovering it.

~Jon Snyderman

Maybe it's a good idea, to run a test label first, so that you can proof that FTP is working in the "normal" situation. Just copy the spool-file content and try it, or make your own test label like:

^XA
^FO10,10^A0N,40,40^FDMY TEST LABEL^FS
^PQ1
^XZ

So if I create a text document called PUT Label.txt
with the fillowing:

^XA
^FO10,10^A0N,40,40^FDMY TEST LABEL^FS
^PQ1
^XZ

and ftp to the Zebra IP how do I get it to print or is there something in the ftp window?

"So if I create a text document called PUT Label.txt"

No crate a text document Label.txt and use the FTP PUT to send it to the printer.

how do I get it to print

ZPL is a print language, you don't send printable data to the printer, but  a set of instructions to build a label

^XA          Open new label
^FO10,10     Positioning to coördinate 10,10
^A0N,20,40   Try the zero scalable font scale width(20) height(40)
^FD          Field Data
^FS          Field separator (end of text)
^PQ1         Print Quantity 1
^XZ          End of label data

Select allOpen in new window

The Code ^XZ indicates that the label is complete and the printer will run this script to print 1 label (as indicated in the PQ command)
Yes ^PQ5 will print 5 labels.

See the ZPLII manual for a complete set of ZPL script commands

To send the labal via ftp, you probably need a userId and Password the default for Zebra is:
UserId = user
Password = password


Have fun :-)

Murph

Murph, I think what he was asking is what initiates the print of the TXT file.  

BC, That is automatically done when the put is complete.  It should some out right away.  If not, close the FTP session and it should print.

~Jon

Ok so I am clear:

1. have a label.txt containing:
      ^XA
      ^FO10,10^A0N,40,40^FDMY TEST LABEL^FS
      ^PQ5
      ^XZ
2. Next time it happens I will ftp to the printer form command prompt and put label.txt

Let me know and I'm waiting for the next report to test

Hi  bergquistcompany,

You better test a label right now, so that you are sure that this test label is working, and you do it the right way.

Yeah we had a printer hang on send and tested the FTP scenario you mentioned and the printer was able to print the labels from the FTP PUT command.  Thus it doesn't appear to be an issue with the printer.  What next?

From the 400, do a WRKOUTQD on the device name and capture that data for us.
Also, if there is an associated device description, do a WRKDEVD to capture that data also.

~Jon

Now we can remove 2 points from the list:

- it isn't Wrong label data (then the printer would not accept a new label)
- it isn't any other printer problem.

Just make a list of the other options and check them one by one I know 4 but there could be more:

- driver
- protocol
- network
- as/400

Do you have network components that use netbios or appletalk?

Attached is all the screen shots from workoutqd and when I do a Workdevd I get the last screen shot after enterin workdevd bfshop1 on the line so maybe I'm entering it wrong but that doesn't work.

So it's not:

- it isn't Wrong label data (then the printer would not accept a new label)
- it isn't any other printer problem.

What to try next?   It doesn't use a driver so how can I determine if it's a network issue or AS400?  I can ftp across the network to the printer when the AS400 queue gets stuck sending.

attachment
Workoutqd.docx

Hi,

If the problem  shows-up, can you check the printer writer status?
WRKWTR WTR(*PRT)

"To get it printing again we have to stop the writer, power off the printer.  
Power on the printer and restart the writer."

You have to power-off the printer? Stop and start the writer isn't enough?

Try to clear the printer buffer by FTP a file tot the printer containing:
~JA

RTM:
Cancel All
Description  The ~JA command cancels all format commands in the buffer. It also cancels
any batches that are printing.
The printer stops after the current label is finished printing. All internal buffers are cleared of
data and the DATA LED turn off.
Submitting this command to the printer scans the buffer and deletes only the data before the
~JA in the input buffer — it does not scan the remainder of the buffer for additional ~JA
commands.

If the printer continues without restarting the writer (and printer) the ~JA indicates that there must be some invalid data in the buffer

Sorry for the delay.   Your settings look good.   This is happening across the board???   Printer itself is staying online?  System sees it unavailable?  I am really starting to lean back towards a physical layer problem but the number of devices confuses that issue.   Is there a common physical characteristic such as building location, attached switch, etc...   The printer may be working when you go to test, but at some point, the 400 saw it go offline.  

When the problem happens, can you try sending the FTP file directly from the 400 vs a PC?

I like Murpheys buffer idea too.

~Jon

How many different type of labels do you print?

Is there any reason to suspect one of the label-types to causes this problem?

If you know what label is send the moment you get this problem, just send that label to an other printer, so that you can see the ZPL coding.

@ murphy2 I will check wrkwtr wtr(*prt) next time it fails.  Is *prt the output queue name?
Yes we do need to power off the printer.  When you say FTP a file would we use the put command as before and have a text file with just ~JA ?  After this try printing another label normally?

No given different locations there are different labels so I can't pin it to one label type.  I could see if next time it fails send to another printer and see if that gets hung on same label.

@jsnyderman Yes it happens intermittently at 3 of our remote branch offices and not with the same printers either.  Printer can receive FTP label when 400 says SENDING and we have to restart the writer and printer.  Can we ftp from within the 400?  I'd be up for giving it a go.

Hi,

No *PRT is NOT the queue name, *PRT is a selection you also can use *ALL , give other info

"Can we ftp from within the 400?  I'd be up for giving it a go."
Yes i'm quite sure, as400-printer connection is an open (online) connection and the problem sounds like a kind of line-drop, FTP is a temporary connection an can be done over and over it's almost impossible to send that on the moment that a line-drop occurs :-)

Have a nice weekend

Ok I will test next time:
1. check wrkwtr wtr(*prt) next time it fails.
2. Try not to reboot printer and see
3. Try the ~JA from PC or AS400 and retry the label
sound like next steps?

Yes,

The ~JA, doesn't matter where it comes from, it wil clear the buffer,
normally I start every label with is. :-)

Been away for a couple of weeks.  Wow, this one has been going on a long time...

We've got thousands of various Zebra printers (and tens of thousands of IP printers in general) deployed across our client base, with several deployments of over 1000 IP printers.  Our success rate in resolving these issues is -very high- even when IBM and the printer vendor have been engaged and been unable to solve the problem - however the level of technical skill required to diagnose the problem can be high, since the problem can be with the sending application, the AS/400 TCP configuration, an intermediate network device, the printer hardware, or the printer firmware.  

I looked back through all the posts, and I didn't ever see a posting of the writer job log.  In my first post, -way- up at the top, I asked for this, but didn't explain in detail what I wanted.  

I didn't mean to look at QSYSLOG, I mean that when the writer is stuck in SND status, to inspect the writer job log:

WRKWTR (printername)
- F24 (More keys)
- F17 (Writer job)
- Option 10 - Job log
- F10 - Display Detailed messages

See if there are any messages indicating the cause of the hang.  Sometimes there will be, sometimes there will not.

If there are any error messages, put your cursor on each one, and press F1 for the extended message description.  Post the extended message description for each message found here.

The AS/400 uses LPR to talk to these printers.  The writer job goes into SND status when it initiates the LPR/LPD session with the remote printer, and stays there until the session completes.  If an expected response never arrives, the writer can hang in SND.

There are only three common things that cause this to happen:

1) An error in the printer's LPD stack that causes it to hang in a non-responding state.  (Update the firmware or engage the vendor for a fix, but first you have to prove it is their firmware).

Also note that LPD and FTP are different daemons running on different ports on the printers print server, so it is possible for the LPD daemon to be hung while the FTP daemon can still process requests.

2) A communications problem that causes requests from the AS/400 or the responses from printer to be discarded or mis-routed.  

(I've seen similar problems from a whole host of network causes, including VPN errors, firewall problems, drops due to excessive inbound/outbound traffic, routing errors, duplicate IP addresses, duplicate MAC addresses, a failing router or switch port, etc.)

3) A TCP problem on the AS/400 - specifically being unable to allocate an LPD port.  This typically only happens on very large systems with hundreds of IP printers, or when there is a conflict for LPR port space with another application.  NETSTAT will help diagnose this, looking to see what is using source ports 256-1024 (the only ports AS/400 LPR can use to initiate an LPR connection).

First we need to isolate the cause of the problem.  Some problems may not be directly fixable (crashing LPD daemon).

Here are some questions and diagnostic procedures that we can use to isolate the cause:

1) Approximately how many LPD printers are configured on your AS/400, in total?

2) When you observe a writer stuck in SND, review the writer job log as I outlined above.  It doesn't always produce a message (or the message may take some time to show up, for example if there is a time-out, so keep checking for about 10 minutes before ending the writer and rebooting the printer).

3) When does this problem happen?  Same time each day?  Random times of day and night?  When one printer gets stuck, does it then start off a wave of stuck printers?  If so, you can start a communications trace and send some test documents to an idle printer in hopes of sticking it, too.

4) Is it a particular document that always sticks?  Is it only large documents, or is document size not a factor?

5) When the problem starts, do a quick communications check by PINGing the printer's IP address (as shown in the DEVD or OUTQD) from the AS/400 (PING <F4><F10>).  Use a large packet - 512 bytes, and send 20 packets.  See if you experience any dropped packets.  If so, you've probably got a network problem.

6) When a printer hangs, use the AS/400 NETSTAT command, option 3, to view a list of all active TCP sessions.  F15 allows you to subset this list to see only LPR/LPD connections.  In the "Remote port range" field at the bottom of the Subset screen, put 515 (this is the port that an LPD daemon listens on) in the "Lower Value" field, and press Enter.

Now the list is restricted to just LPR/LPD sessions.  Print the list using F6 and post here.

Next, look down the Remote Address column and find the hung printer.  Use option 5 to display the connection status information.  Print using F6 and post here.

General advice

Probably fixes 50% of these types of issues if you are out of date on updates and fixes, and yet it is probably the most frequently-ignored advice I give:

1) Make sure your printers are running the latest Zebra firmware.  Latest firmware for your model is V60.13.0.13Z .

http://www.zebra.com/id/zebra/na/en/index/drivers_downloads/firmware/firmware_for_printers.html

2) Create a problem log, and carefully log the date, time, writer, IP address, document length, document type, min/max PING time, and number of LPR/LPD sessions showing in NETSTAT for each hang.  We want to see if a pattern emerges.

3) Make sure your AS/400 is patched with the latest cume PTF, and group PTFs for printing and TCP/IP:

http://www-947.ibm.com/systems/support/i/fixes/

AS/400 LPR Ports info:

http://www-01.ibm.com/support/docview.wss?uid=nas1995ddeb518a024d886256bac005113e2

- Gary

Hi Gary,

Yes we notices that you have been away for a couple of weeks, for the first time in months I was able to earn some points here, but I have the feeling we have to split (again) on this one

Welcome back

You guys rock!  I will make sure to try these suggestions next occurrance!  I agree it's gone on too long, but finally feel we're making progress when in the past it's been every vendor pointing at the next and nobody really ever said try this!  I'll gather all this and post
my findings

For whatever it is worth, this is NOT an uncommon type of problem.  I see above that others have experienced it, and we've had similar incidents in a number of customer networks over the years.

Troubleshooting can be complex, and since it can cross system and network boundaries, and multiple devices and vendors are often involved, it can be tough to get a resolution unless you can troubleshoot the whole thing end-to-end.

- Gary

I can test everything end to end and have your list.  Today we had an intermic that got stuck.
I was so excited because I thought we could capture enough to get the next step.

1. WRKWTR WTR(*PRT) - the printer doesn't show here because these are client access 5250 printers.  The ones that are hanging are remote as in to restart the writer we type: strrmtwtr.

2. In this case today it was stuck because a report was sent to the printer (not a label).
In most cases it's been a label so we'll wait for the next one to happen and proceed with the ~JA test and other gatherings as requested.

I appreciate you both sticking with me to weed through this.  Indeed the vendors have all pointed fingers and unable to help so you have both been exceptional in getting us closer with things to try and weed out!  THANK YOU!  I look forward to the next hanging Zebra :)

FYI to your question it happens intermittently on Zebra and Intermic printers in different branch offices printing different labels so we haven't been able to say it's a specific label.  Will have more to report next issue.

WRKWTR WTR(*PRT)

That's not quite the command.  What is the name of the hung writer?  Let's say it is called IMSTUCK.  Issue this command:

WRKWTR IMSTUCK

You can also get there with WRKWTR *ALL, and page through the list of you prefer.

This is the first time you've mentioned a problem with Client Access connected printers.  Just to be clear, you mean that the printer is configured using a Client Access (IBM iSeries Access / IBM i Access) printer emulation session running on a PC or server?

Client Access printer emulation doesn't use LPR/LPD - they use the TN5250E protocol (assuming this is a TCP/IP environment).

Looking at the remote OUTQ description you posted up at the top of the thread (SHIP2), that is an IP printer directly attached to the network, and the AS400 is communicating with it directly using LPR/LPD.

Today's problem was with a client-access connected printer.

Interesting.  Well, same basic troubleshooting procedures apply.

- Gary

Hi Gary,

Ok I had the command wrong and this time don't think it would have mattered as it was a report incorrectly sent to the label printer, but I will try that command next time.

No I thought the list that came up was for Client Access connected printers and the issue is on the ones NOT configured using Client Access.  I am clear now and was just using the wrong command.  Again it is on the printers NOT configured using Client Access.  

You are correct it is IP Printer.  I misunderstood the command and sorry for causing confusion. I will grab the information on the next hang.

Today's problem was a IP Printer as well, but due to a report.  Usually I do note it's a label.  I just misspoke saying the command was for Client Access printers and mine wasn't, but I didn't understand the command.  I am clear now thank you and again it is only IP Printers I'm troubleshooting.

No problem!  This can really get confusing.

- Gary

Actually I'm learning a TON and am so excited that we're weeding things out and not fighting vendors who finger point.  I'll gather more next hang up

Hi bergquistcompany,

you write : "these are client access 5250 printers",

do you mean that they are connected to a PC with a 5250 printer session?

no murphy I was confused when I entered the command it listed the 5250 printers and I was saying mine wasn't in there because it was an IP printer.  I shouldn't have brought up 5250 it was my mistake for getting the command wrong.  They are all IP Printers having the issue.

waiting to get one tested this week.  Sometimes they can reset the writer depending on permissions.  Have informed everyone to go to me so I can test first.

Ok here is what I have see attached screen shots and additional steps.  The ftp of the ja.txt containing ~ja didn't work though given this disconnect was an intermic printer, but the same symptoms apply whether zebra or intermic.  Hope this helps with more.
Disconnect.docx

If it was NOT a zebra printer, I can imagine that ~JA didn't work,
Zebra printers are working with ZPL or ZPL2 (Zebra Printer Language)
while Itermic printers use IPL (yes correct Imtermic Printer Language)
and they are not compatible:

ZPL looks like:

      ^XA
      ^FO10,10^A0N,40,40^FDMY TEST LABEL^FS
      ^PQ5
      ^XZ

Select allOpen in new window

While IPL looks like:

<STX><ESC>C<ETX>
<STX><ESC>P<ETX>
<STX>THIS IS THE SAMPLE LABEL<CR><ETX>  
<STX>SAMPLE<ETX>
<STX><ETB><ETX>

Select allOpen in new window

In your example :
10.80.5.17 is your printer server I assume?
Didi you ever try to connect the writer directly to printer address 10.10.4.1

The AS400 is 10.10.4.1
The intermic in this case was 10.80.5.17

Ok I have the suggestions and will gather on the next hang.  Thank you so much you are showing me a great deal and I am very appreciative of all your expertise.

Not enough information to confirm an answer.

@hdhondt,

Not agree, if you see authors last comment, we DO give him a load of inforamtion to solve his problem, I sugest to split the points between  Gary_The_IT_Pro and  murphey2

Hey  bergquistcompany,

Please give us an update, we like to know the status of your problem :-)

We've been working on this one a long time, and it has only been a few weeks since the last activity.

The OP explained that this is an intermittent problem - we need to leave this open to allow time for the problem to happen again and for the poster to perform the lengthy list of troubleshooting requested.

- Gary

Please do not delete this question.  Gary and Murphy have offered so much information I want to reward them but we've been going through a series of training steps with our users to report this specific issue as they ahve not always been channeling it our way.  I would like to leave it open a few more weeks to give us a little more time as it is intermittent and I don't want to lose touch of the expertise I have found with these two outstanding gentlemen.

I can't get near the help with IBM.

Hi bergquistcompany,

Give us a weekly update of the problem (ans solution) status, that is good for us and keep the question open.

What is the current status?

I contacted EE's customer service and asked them to re-open this.  They graciously complied.  Thanks.

Apparently, I pressed the "Submit" button instead of the "Object" button and that is why it went ahead with the deletion.

- Gary

Status: Pending recommendation to mgt to get PTF updated and will let you know next week where we are at so we can see if this resolves issues

@Gary What is the command to check PTF level.  Our system admin for the AS400 says they are up to date and have been for 2 weeks.  I would like to check this and confirm from your expertise?

I reviewed the output you provided from WRKPTFGRP command (I think that was posted on the other question you opened related to this problem).  

Based on the WRKPTFGRP While your Cumulative PTF level was current, your Group PTF levels for TCP/IP and Printing were behind, as I explained previously.

Group PTF packages sometimes contain product-specific fixes that are not included in cumulative PTF packages, or are newer than the most recent cumulative release.

http://www-947.ibm.com/systems/support/i/fixes/guide/maintstrat.html

- Gary Patterson

@Gary and @ Murphy - you two are amazing and I can't thank you enough for all your time and attention.  We are going to apply the PTFs this weekend and I would have never been able to come up with a next step for management without your help!

You are an asset to this site and I will close this and award you both for your time and if after the application I run into further issues I will create a new post related to what is going on, but after IBM and other administrators pointing fingers you helped me find a path to follow and I can't say thank you enough!!

Happy to help.  Feel free to open a new question if after applying PTFs you still experience problems.  

- Gary