How to navigate multi-file python program such as bitbake?

naseeam
naseeam used Ask the Experts™
on
BitBake is a Python Program.  I downloaded BitBake from  https://github.com/openembedded/bitbake.  I selected branch version 1.40.0 and downloaded the zip.  
At least some of the source code is in bitbake-1.40.0/bin  and  bitbake-14.0.0/lib folders.

I can insert print statements to navigate source code.  Is there alternate way to navigate python program such as bitbake?
Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
What do you mean exactly with navigate?

You might use Visualstudio Code or Pycharm to navigate through the source code. (Both run on Windows and Linux sytems)

For debugging and stepping through the source code you can use the python debugger

Author

Commented:
>> What do you mean exactly with navigate?
I mean single step python code.  Insert breakpoints.

>> use python debugger
I'm using Linux virtual box.  I've never used python debugger.  How to get started?

>> You might use Visualstudio Code or Pycharm to navigate through the source code.
BitBake Python program is used to build embedded Linux distributions.  I don't think it'll run in windows environment?
Just for info. Pycharm and visualstudio code run both on Linux and Windows, but that's just for viewing the source code.


If you can edit the files (which you can as you were able to add print statements),
then just add following line at the place from which you want to start stepping through the source code.

import pdb;pdb.set_trace()

Open in new window


When you run bitbake and it hits above line it will automatically enter python debugger.


You can run a slightly improved python debugger by installing pdb++ (pip install pdbpp)  (see https://pypi.org/project/pdbpp/ )
Acronis in Gartner 2019 MQ for datacenter backup

It is an honor to be featured in Gartner 2019 Magic Quadrant for Datacenter Backup and Recovery Solutions. Gartner’s MQ sets a high standard and earning a place on their grid is a great affirmation that Acronis is delivering on our mission to protect all data, apps, and systems.

nociSoftware Engineer
Distinguished Expert 2018

Commented:
vim can be language sensitive for python as well which might help editon on small systems.
On severly limited system this might not be an option.
David FavorFractional CTO
Distinguished Expert 2018

Commented:
https://docs.python.org/3/library/pdb.html provides a pure Python debugger.
David FavorFractional CTO
Distinguished Expert 2018

Commented:
Consideration: There are many parallel job task managers, so if BitBake fails to work, likely faster to just pick a working task manager than debugging + fixing BitBake.
David Favor has a valid point: debugging / fixing bit bake might be painful. On the other hand. if you have a bit bake project you might have to solve your issue as switching is not a real alternative.

Though I gave you the answer how you can single step through bitbake, the question is whether this will help you identify the issue that you have.

Though I don't know how many bitbake experts you will find (I was using it a once over 10 years ago and didn't use it since) it might be faster if you can characterize your problem and discuss it in a bit bake group.

Singlestepping through a huge application can be very painful and time consuming

Author

Commented:
>> There are many parallel job task managers
What does this mean?  Do you mean phython code I'm stepping through has many tasks running concurrently?  If yes, I have only studied python up to and including classes.  Are there also periodic tasks in python?
Who is  managing job tasks, Linux OS?

>> so if BitBake fails to work, likely faster to just pick a working task manager than debugging + fixing BitBake.
I don't understand this statement.  How to  understand "task manager" concept in python?

>> debugging / fixing bit bake might be painful
I agree.  BitBake appears to be working fine.  It's from a stable branch.
I'm just stepping through the source code to enhance my python skills and hopefully it will be helpful when I create custom embedded Linux distributions.

>> Though I gave you the answer how you can single step through bitbake, the question is whether this will help you identify the issue that you have.
Yes, the question has already been answered well.  There is no issue with bitbake.

>> Singlestepping through a huge application can be very painful and time consuming
I agree.  I hope I'll get some understanding on how a large python project is coded.
nociSoftware Engineer
Distinguished Expert 2018

Commented:
If the embedded system runs linux..., why not first try on a system capable of useful debugging.
And when all wrinkles are ironed out try on the embedded system.
You may need some stubs for missing hardware etc.
OK great to know, that there are no issues with bitbake.

I worried that you try to find bugs in bit bake by going through the source code and by stepping through its execution.

bitbake is a quite huge project.


If your goal is to create custom embedded linux distributions with bit bake, then I think a good strategy is to learn basic python with some tutorials and then dive into bit bakes documentation ( https://www.yoctoproject.org/docs/3.0/bitbake-user-manual/bitbake-user-manual.html ) . I think in your case it's more important to understand bitbake's philosophy and structure than learning all about python.

Then try to play with bitbakes recipes and configuration and whenever you encounter python code, that doesn't make sense to you try to learn / read about it.

What's a little annoying with learning things like bitbake for created embedded linux packages is, that a full cycle (from recipe to a file system for your embedded linux) takes quite some time.

Author

Commented:
Thank you for setting up python debugger for my project.

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial