mmadia ([info]mmadia) wrote in [info]bezilla,
@ 2008-03-15 16:32:00
Previous Entry  Add to memories!  Tell a Friend!  Next Entry
Re-ordering *.so during build time
Earlier this month, there was a discussion on the [haiku-development] mailing list about repackaging Firefox to allow a more native feel.

I've been working on getting mozilla's build system to produce this at compile time.
... And significant progress has been made!


I found the sections of code in the various Makefile.in's, and rules.mk that are responsible for either :
directly copying the *.so into $(DIST)/bin
symlinknig *.so into $(DIST)/bin
and creating and copying *.so.stub for those *.so's

a quick and dirty test confirms that those *.so's can be placed into $(DIST)/bin/lib to allow direct launching of firefox-bin

There's still some more work to do:
find and remove support for the launching shell script, 'firefox'
find and renove '-bin' from the binary name
cleaning up code for patch submittal.


a few questions:
should the corresponding *.so.stub's be removed?
how important is it to have 'libxpistub.so' in ./add-ons instead of ./lib ?
('libfreebl3.so','libnssckbi.so' can easily be relocated to ./add-ons )

(Post a new comment)


[info]fyysik
2008-03-16 09:25 am UTC (link)
>find and renove '-bin' from the binary name

there is ifeq OS_NAME_HERE somewhere (saw it once) which defines which name binary will have.

>should the corresponding *.so.stub's be removed?

For Haiku and latest Zeta versions - probably yes.
No - for BeOS and early Zetas

(Reply to this) (Thread)

[info]tqh
2008-03-16 12:55 pm UTC (link)
The stubs shouldn't even be built.

There are two things to watch out for with reordering. Making sure SSL works. And that the addons-memory isn't used up. The addons-memory is problematic to test, but running SoundPlayer and other apps that use addon memory is afaik the way to see this problem. Try to make sure only those libs linked directly to firefox-bin is in lib/ you can see this by objdump -X | grep NEEDED (or was it x).

(Reply to this) (Parent)

[info]tigerdog
2008-03-16 03:21 pm UTC (link)
I've been running without the stub files under BeOS BONE and R5.0.3 ever since tqh published his findings. Fyysik, since I don't know why the stubs existed in the first place, I don't know if I'm testing the scenario that requires them.

SSL always works fine using this arrangement - as long as you move the files to their final location BEFORE the first run. If you run Firefox then try to move its folder, SSL doesn't work.

I'll try running SoundPlayer in multiple instances with Firefox and see what happens.

(Reply to this) (Parent)(Thread)

[info]fyysik
2008-03-16 03:34 pm UTC (link)
BeOS R5, BeOS R5 BONE have bad low limit for memory size provided for add-ons - components to load at runtime - those real *.so libs. Both per app and per whole system.

https://bugzilla.mozilla.org/show_bug.cgi?id=51655

That didn't allow to port Mozilla to BeOS for long time, until workaround was found.

Use fake add-ons (for those limit is higher), whose only functions is to load *.so-s after its own loading.

As firefox uses bit less add-ons than seamonkey, it can start without stubs. But as add-on space is "shared" for whole systems, users frequently have hit that limit - with lot of Tracker add-on installed, or with such apps as Frans Nispen's paint/imaging studio.

Frans did corrction for latest Zeta kernels, IIRC, enlarging that space, but for R5 it isn't the case.

(Reply to this) (Parent)(Thread)

[info]mmadia
2008-03-16 04:05 pm UTC (link)
as the *.so.stub's in question are normally placed in firefox/, could they be relocated with their respective *.so to ./add-ons and ./lib?

(Reply to this) (Parent)

[info]tigerdog
2008-03-16 06:21 pm UTC (link)
It may also be that since we're now making static builds for distribution, we're creating fewer .so files in the first place. I know there are a lot more .so and .stub files when I create a dynamic build.

I wonder if we could create static builds for R5 and BONE and ask people to beat on them before we commit to this change.

(Reply to this) (Parent)(Thread)

[info]fyysik
2008-03-16 06:36 pm UTC (link)
I'm unsure if number of libs with smaller size occupies much more add-on space than one big lib containing all those components which are in small libs. There may be some benefit there though. But for other side, big lib will never be unloaded at runtime, as, with big probability, some component from there is required at any moment, while for small libs such situation may be possible.

(Reply to this) (Parent)(Thread)

[info]tqh
2008-03-16 08:13 pm UTC (link)
I think a static build would probably have a lot of 'addons' linked to the firefox-bin which should help reduce the use of addon-memory. At least judging from the size of the executable.

(Reply to this) (Parent)

[info]tigerdog
2008-03-17 04:02 pm UTC (link)
Here's an example:
c++ -o nsBookmarksService.o -c -DMOZILLA_INTERNAL_API -DOSTYPE=\"BeOS6.2\" -DOSARCH=\"BeOS\" -DBUILD_ID=0000000000

This was from the Zeta 1.51 beta release, hence the "6.2". BONE, Dano and R5 all report the expected version. I can produce a list if that helps...

(Reply to this) (Parent)

[info]kymigosu
2008-07-10 11:11 pm UTC (link)
Every time you try to move one in Firefox the windows closes and you’re back to square one. IE isn’t much better because it routinely fails to update deleted in the same window.

(Reply to this) (Parent)

[info]darianegyb
2008-07-11 01:00 am UTC (link)
I am going to try other software to see if it happens with that. Just wondering if this is a known issue since k3b was fine before.

(Reply to this) (Parent)

[info]tigerdog
2008-03-17 02:56 pm UTC (link)
is it possible to test on the -DOSTYPE flag to determine which version to set? I note this is determined (correctly) during the build.

(Reply to this) (Parent)
figured out the packager
[info]mmadia
2008-03-16 03:21 pm UTC (link)
the packager is now aware of these changes and archives the distribution properly. in short, browser/installer/Makefile.in now does an OS_ARCH, BeOS test and grabs a new beos/packages-static file

here's the affected source tree files
compilation:
config/rules.mk
nsprpub/lib/ds/Makefile.in
nsprpub/lib/libc/src/Makefile.in
nsprpub/pr/src/Makefile.in
security/manager/Makefile.in

packaging:
browser/installer/Makefile.in
browser/installer/beos/packages-static

the compilation changes affect all mozilla products.
the packaging changes need to be applied to each product's installer folder.
Another benefit to this, we can now remove any file from the archived distribution, such as run-mozilla.sh

(Reply to this)

status
[info]mmadia
2008-03-17 03:35 am UTC (link)
At least for firefox, everything is working properly;
building, packaging, and running natively.
( note: i've still yet to put this to the add-on test )

The respective *.so.stub's are placed in ./lib,
even though i currently don't have any *.so.stubs that belong in ./add-ons, a rule was added for good measure.

'firefox', the script is no longer included in the archive.
'firefox-bin', is renamed 'firefox'
Currenty, if the value for$(PROGRAM) is not the same as $(MOZ_PKG_APPNAME)
then the binary will get stripped.

personally, i'm content with 'firefox' instead of 'Firefox'
especially as there's more important issues at the moment.

I'm building Seamonkey, Thunderbird, and Sunbird to see how they're affected and what needs to be done.

Once all products build and package, i'll clean up the code and submit patches. Of course, if you'd like them before that just let me know :)

(Reply to this) (Thread)
Re: status
[info]tigerdog
2008-03-17 01:57 pm UTC (link)
maybe I'm being silly, but what would people think of calling the executable "Firefox" instead of "firefox"? The Mozilla foundation is pretty pick about naming and spelling for their "official" builds and insist it should be spelled with an upper-case "F" and no other capitals.

(Reply to this) (Parent)(Thread)
Re: status
[info]mmadia
2008-03-17 11:04 pm UTC (link)
for those official builds, does the build system use an upper-case "F" or are you manually changing that before packaging?

(Reply to this) (Parent)(Thread)
Re: status
[info]tqh
2008-03-18 07:58 am UTC (link)
I'd prefer uppercase, but if it's complicated it's something we can leave for later.

(Reply to this) (Parent)
Re: status
[info]tigerdog
2008-03-18 01:30 pm UTC (link)
For the "official" builds, I leave things as they are - lower case. For personal use, I reorder a Zeta build, delete "firefox" (the script) and rename "firefox-bin" to "Firefox". I also rename the folder to "Firefox" because it matches what was distributed with Zeta.

(Reply to this) (Parent)
firefox, seamonkey, thunderbird, and sunbird = OK
[info]mmadia
2008-03-18 03:06 am UTC (link)
all four products are being built in a re-ordered state and are capable of launching.

in all, 12 files needed to be editied, a folder and a file needed to be added to browser/installer

tomorrow i'll pull a clean tree, apply the changes and create a patch.
should it be one large one since it's all for one commit
or each individual file?

afterwards, i'll try building the oddball products; composer and xulrunner. can we ignore minimo and macbrowser?
and what's 'standalone' http://mxr.mozilla.org/mozilla1.8/source/configure.in#4457 ?

(Reply to this) (Thread)
Re: firefox, seamonkey, thunderbird, and sunbird = OK
[info]tqh
2008-03-18 07:59 am UTC (link)
Excellent, is the Firefox reordering different from my old blog-post?

(Reply to this) (Parent)(Thread)
Re: firefox, seamonkey, thunderbird, and sunbird = OK
[info]mmadia
2008-03-18 10:29 pm UTC (link)
yes, slighty, libxpistub.so goes to ./libs
any *.so.stubs are moved with their *.so

here's a breakdown of what's handled by what

./add-ons
( each handled in seperate sections of security/manager/Makefile.in)
libfreebl3.so, libnssckbi.so

./lib
(handled by the same section in config/rules.mk)
libmozjs.so, libmozjs.so.stub
libxpcom.so, libxpcom.so.stub
libxpcom_compat.so, libxpcom_compat.so.stub
libxpcom_core.so, libxpcom_core.so.stub
libxpistub.so, libxpistub.so.stub

(handled by 3 seperate Makefiles under nsprpub/ )
libnspr4.so, libplc4.so, libplds4.so

(handled by security/manage/Makefile.in)
libnss3.so, libsmime3.so, libsoftokn3.so, libssl3.so

Seamonkey and Thunderbird have two extra files in ./lib,
(handled by directory/c-sdk/ldap/libraries/[libldap, libprldap]/Makefile.in )
libldap50.so, libprldap50.so

(Reply to this) (Parent)(Thread)
Re: firefox, seamonkey, thunderbird, and sunbird = OK
[info]tqh
2008-03-19 12:41 pm UTC (link)
One more thing that I came to think of, test also with new Mozilla profiles. It's great if we can get this into the tree.

(Reply to this) (Parent)(Thread)
Re: firefox, seamonkey, thunderbird, and sunbird = OK
[info]mmadia
2008-03-19 04:16 pm UTC (link)
sum : they all work (SM, FF, TB, SB)

the profile directories ~/config/settings/Mozilla and /Thunderbird were deleted.

launched each app successfully.
they displayed in deskbar,
created the profile directory,
disappeared from deskbar,
relaunched and displayed properly.

on a side note, FF and SB have their profiles in /Mozilla,
is there any reason why TB isn't?

(Reply to this) (Parent)
Re: firefox, seamonkey, thunderbird, and sunbird = OK
[info]tigerdog
2008-03-19 12:04 am UTC (link)
I'll be happy to apply and test the patch. This is exciting!

(Reply to this) (Parent)

Create an Account
Forgot your login or password?
Login w/ OpenID
English • Español • Deutsch • Русский…