Opened 13 years ago

Closed 13 years ago

Last modified 13 years ago

#618 closed defect (worksforme)

problem linking newish ncurses & fontconfig

Reported by: sci-fi@… Owned by: diego@…
Priority: very important Component: build system
Version: HEAD Severity: blocker
Keywords: Cc: sci-fi@…
Blocked By: Blocking:
Reproduced by developer: Analyzed by developer:

Description

Hi,

I finally got some time to wipe out my workarounds on
compiling ffmpeg & mplayer on OSX/Tiger with latest XCode
2.4.1 and other SDKs that came out this week, to see how it
fares with latest svn. Not good I'm afraid, still needs my
workarounds.

First thing is that the detection for ncurses & fontconfig
are found but later on the linker can't find a ton of symbols:

[...]
make[1]: Leaving directory
`/Volumes/MaxFire3/Projects/svn.mplayerhq.hu/mplayer/libmenu'
cc -o mplayer mplayer.o m_property.o mp_msg.o asxparser.o codec-cfg.o
cpudetect.o edl.o find_sub.o m_config.o m_option.o m_struct.o parser-cfg.o
playtree.o playtreeparser.o spudec.o sub_cc.o subreader.o vobsub.o unrarlib.o
mixer.o parser-mpcmd.o subopt-helper.o libvo/libvo.a libao2/libao2.a
input/libinput.a libmenu/libmenu.a libmpcodecs/libmpcodecs.a libaf/libaf.a
libmpdemux/libmpdemux.a stream/stream.a libswscale/libswscale.a osdep/libosdep.a
libavformat/libavformat.a libavcodec/libavcodec.a libavutil/libavutil.a
libpostproc/libpostproc.a -Wl,-read_only_relocs,suppress
/usr/local/lib/live/liveMedia/libliveMedia.a
/usr/local/lib/live/groupsock/libgroupsock.a
/usr/local/lib/live/UsageEnvironment/libUsageEnvironment.a
/usr/local/lib/live/BasicUsageEnvironment/libBasicUsageEnvironment.a
-lstdc++ -lz -lncurses -lm -lgif -L/usr/local/lib -ldvdnav
-lpthread -framework Carbon -framework QuickTime? -framework CoreAudio? -framework
AudioUnit? -framework AudioToolbox? -framework Cocoa -framework QuartzCore?
-framework OpenGL -lcdda_interface -lcdda_paranoia -lxvidcore -lx264
-lmp3lame -lvorbis -logg -ldts -lmad -lvorbis -logg -ldv -I/usr/local/include

-L/usr/local/lib -ltheora -logg -lspeex -ltwolame -lfaac -lmpcdec -llzo

-lpng -lz -ljpeg -liconv libfaad2/libfaad2.a mp3lib/libMP3.a
liba52/liba52.a libmpeg2/libmpeg2.a libass/libass.a -L/usr/local/lib -lfreetype
-lz -L/usr/local/lib -lSDLmain -lSDL -Wl,-framework,Cocoa -L/usr/local/lib
-lcaca -lcucul -lGL -lXv -lXxf86vm -lXinerama -L/usr/X11R6/lib -lXext
-lX11 -L/usr/local/lib -lesd -laudiofile -lm -ljack -framework OpenAL
/usr/bin/ld: warning suggest use of -bind_at_load, as lazy binding may result in
errors or different symbols being used
/usr/bin/ld: Undefined symbols:
_FcConfigSubstitute
_FcDefaultSubstitute
_FcFontMatch
_FcInit
_FcNameParse
_FcPatternGetBool
_FcPatternGetString
_UDFFindFile
_tgetent
_tgetnum
_tgetstr
_FcConfigAppFontAddDir
_FcConfigGetBlanks
_FcConfigGetCurrent
_FcDirCacheValid
_FcDirSave
_FcDirScan
_FcFontSetCreate
_FcGetVersion
_FcPatternAddBool
_FcPatternAddInteger
_FcPatternAddString
_FcPatternCreate
_FcPatternGetInteger
_FcStrSetAdd
_FcStrSetCreate
nc_outch
nc_screen
_cur_term
_curscr
_putp
_tparm
_tputs
nc_keypad
nc_timed_wait
nc_update_screensize
_reset_shell_mode
nc_add_to_try
nc_flush
_keyok
_tigetstr
_def_prog_mode
_stdscr
_COLS
_LINES
nc_getenv_num
nc_handle_sigwinch
nc_set_screen
nc_set_tty_mode
_baudrate
_cbreak
_setupterm
_typeahead
_newscr
_TABSIZE
_unctrl
nc_get_tty_mode
_erasechar
_killchar
_noraw
nc_safe_strcat
nc_safe_strcpy
nc_str_copy
nc_str_init
nc_str_null
_curs_set
_has_ic
_reset_prog_mode
nc_alloc_screen
nc_init_acs
nc_screen_chain
nc_set_buffer
_def_shell_mode
_del_curterm
_set_curterm
_flushinp
nc_doalloc
nc_acs_map
symbol _DrawText used from dynamic library
/System/Library/Frameworks/ApplicationServices?.framework/Versions/A/Frameworks/QD.framework/Versions/A/QD(QD.o)
not from earlier dynamic library /usr/local/lib/libgif.4.dylib(gif_font.o)
collect2: ld returned 1 exit status
make: * [mplayer] Error 1
make: Leaving directory `/Volumes/MaxFire3/Projects/svn.mplayerhq.hu/mplayer'

(1) To clear-up the ncurses part of this problem, we need
to add -ltinfo together with -lncurses -- IIRC they split
these libs apart around 5.1 or 5.2 of ncurses, and Apple's
Tiger suddenly went with a later version than that (presently
5.4.20040208). Don't feel lonely, I've had to add -ltinfo
to *many* open source projects that detect/use ncurses.

(2) To clear-up the fontconfig part of this problem, we
need to add -lfontconfig. I don't know why this isn't done
when mplayer's configure detects it.

(3) Now the third problem is that I can't find a way to
specify using my own-built later versions of these libs in
/usr/local/{lib,include} since mplayer's configure script
has no way to specify/override their detected locations. So
as the above linkcmd seems to indicate, it'll use the
vendor-supplied ones in /usr (I think?). We should find
some way to add -L/usr/local/lib (or whatever the developer
wants) ahead of the -lstd++ there, or even earlier inside
the cmd. Where the -Wl,-read_only_relocs,suppress is shown
might be the right place -- this string comes from my shell
env-var LDFLAGS at configure time. Setting LDFLAGS is not
a guarantee for any project to override the first -L
location, though. Too bad ncurses doesn't use pkg-config ...
... but fontconfig does, yet mplayer's configure didn't seem
to pull out the needed info, and my own built & installed
version does have correct info:
# pkg-config --libs fontconfig
-L/usr/local/lib -lfontconfig
# pkg-config --modversion fontconfig
2.3.95
# pkg-config --version
0.21

(4) The last remaining missing symbol is _UDFFindFile which
I have been discussing in another thread on the -devel list,
let's not worry about it here, here I want to only fix missing
symbols for ncurses & fontconfig and also to make sure mplayer
uses my /usr/local properly.

I know if we need to add something like
--with-ncurses{lib,dir}=foo
will disrupt the build system quite a bit. But it should be
easy to try fixing the fontconfig problem. I'll try
whipping up some patches, and will stay tuned here.

p.s. I reported this for the OSX platform, but it may well
occur on all other *ix-based systems that use ncurses.

Thanks for any help.

Attachments (1)

mplayer_configure_fontconfig.patch (814 bytes) - added by sci-fi@… 13 years ago.
let pkg-config have first-dibs on detecting location of fontconfig libs etc.

Download all attachments as: .zip

Change History (11)

Changed 13 years ago by sci-fi@…

let pkg-config have first-dibs on detecting location of fontconfig libs etc.

comment:1 Changed 13 years ago by sci-fi@…

comment:2 Changed 13 years ago by sci-fi@…

Proposed patch to fix the fontconfig part of this bug -- please find attached
file named 'mplayer_configure_fontconfig.patch'.

This patch will re-order the way configure detects fontconfig. Presently the
configure script seems to be bass-ackwards. My way of thinking is to try the
automatic method first e.g. test for pkg-config. Then try the most complicated
set of libs next e.g. cc_check -lfontconfig -lexpat -lfreetype. Finally test
for the simplest case e.g. -lfontconfig.

With the patch, the pkg-config logic has a chance ("first dibs") to properly
fill-in the _inc_extra and _ld_fontconfig vars, which are already passed on to
the Makefile. Other tests here did not set _ld_fontconfig.

Also if one forces --enable-fontconfig, configure does not fill-in _any_ of
those related vars at all. This was my initial case when I opened this bugreport.

There may be many other such places in configure that have bass-ackwards logic
like this. ;) Maybe I can try spotting them while I remove my other workarounds.

For the ncurses part of this bug, it's going to be quite a bit more complicated. ;)

comment:3 Changed 13 years ago by sci-fi@…

  • op_sys changed from Mac OS X to All
  • Priority changed from important to very important
  • rep_platform changed from PPC with Altivec to All
  • Severity changed from major to blocker

Raising this to 'blocking' status and to apply to 'all' systems.

The configure script doesn't have enough test logic to know it needs to include
both -lncurses and -ltinfo on the newer versions of ncurses when these libs were
split.

But the older versions of ncurses will work okay with just/only -lncurses, since
the modules were linked together into that one lib back then.

I'm thinking we'll need (1) a version test in configure, and/or (2) a test for a
module that's been known to be moved into libtinfo. The original missing
symbols list I provided here should have a good clue for the latter. ;)

Also, we still need a way to let configure & Makefiles know to use ncurses/tinfo
from -L/usr/local/lib (or wherever) ahead of other locations. (I wonder if the
ncurses project owner would be open to adding pkg-config support there? ;) )

Right now, the only workaround I can find is to specify
LDFLAGS="-L/usr/local/lib -lncurses -ltinfo"
so that mplayer's build system can get these early ahead of others into the
linkcmds (--extra-libs & other parms are sometimes still too late).

comment:4 Changed 13 years ago by reimar

MPlayer should always try to find the minimum number of libs needed, thus
reordering is not the correct fix. The behaviour of the --enable options is
exactly as intended and configure warns about it in multiple places.
If --extra-libs does not work that is a separate bug and should be report as
such (in a different bugzilla entry obviously).
For the ncurses problem (unless caused by --enable, too) please provide the
configure.log part of this, because for these newer ncurses versions MPlayer
should link against (and only against) tinfo.

comment:5 Changed 13 years ago by sci-fi@…

(In reply to comment #4)

MPlayer should always try to find the minimum number of libs needed, thus
reordering is not the correct fix. The behaviour of the --enable options is
exactly as intended and configure warns about it in multiple places.

Please pardon my shouting, but I MUST make a very important observation with
your response.

MPlayer ought to find the CORRECT libs needed, NOT JUST ONLY THE MINIMUM.

This has been the case of any other open project I've been able to compile on
OSX here. When the pkg-config files are available, those must be chosen FIRST.

Otherwise they are of no use, and MPlayer is ACTIVELY IGNORING THEM ON PURPOSE,

a sinister purpose IMO.

Because of this ill way of thinking IMO, I must therefore leave my configure
patches in-place for my compile of MPlayer. I guess I will need to think about
raising this concern in the public arena e.g. -devel list, then, if I cannot
convince anyone via this bugreport otherwise.

If --extra-libs does not work that is a separate bug and should be report as
such (in a different bugzilla entry obviously).

One should not need --extra-libs just to FORCE obeying the pkg-config details
that the configure script presently refuses to do FIRST!!!

Also think about this, please: What is the purpose of installing different
versions of the same projects into /usr/local, /opt, or even $HOME/{bin,lib},
etc., if one cannot point to THOSE places FIRST for TESTING? The pkg-config
files help point other projects to those newer versions, also, if one's env-vars
are set correctly. MPlayer is bypassing THIS WHOLE SCHEME to become a VERY
*IX-UNFRIENDLY PROJECT IMO!!

For the ncurses problem (unless caused by --enable, too)

Huh??? Where is --enable-ncurses anymore in MPlayer's configure script (from
svn)??? A grep of my whole local svn tree does not show 'enable-ncurses' nor
'enable-curses'. Therefore, the only way it'll enable ncurses ATPT is to detect
it. Once it does this, I have shown the outcome of it with a recent ncurses.

please provide the
configure.log part of this,
because for these newer ncurses versions MPlayer
should link against (and only against) tinfo.

Both -lncurses and -ltinfo are required: all my compiles with other projects,
including this one, have shown this to be true over the past year or two (a
minimum clue as to how long ncurses has had this changed).

The proof of this is already apparent: clearly configure is detecting tgetent in
-lncurses ahead of -ltinfo -- do you see -ltinfo anywhere in my initial log
above? So there's one module in -lncurses that _will_ be required. The proof
of needing -ltinfo also is already apparent: look at the missing symbol list I
showed in the same initial log there. My configure.log file will not show this
as clearly! As I already initially mentioned, _adding_ -ltinfo _with_ -lncurses
is the only way to fix this!

I mentioned Apple's provided version of ncurses in the initial report above;
this lives in /usr. In my /usr/local I have built the latest 5.5 with recent
patches designed to help work with OSX better (details mentioned in their
changelogs; patchlevel here is 20061014 as I write this). It's been a struggle
to get various projects to USE THIS VERSION INSTEAD, mainly because of the same
kind of ill thinking I am reading here IMO, and some due to the way OSX works
(such as: a case-sensitive filesystem is required to use DYLD_LIBRARY_PATH, as
Apple's purposely-misspelled frameworks for libGIF etc. do not have the expected
modules as the original libgif etc., for reasons I hope are readily apparent,
and I might write about this later in the public forums). I'm also really
hoping other platforms can get the newer versions to verify this problem, I do
not know why no one else has already seen it.

...

I'm sorry to see such responses of ill thinking like this IMO. I had hoped to
start officially submitting the changes I've had to apply over the years, just
as I mentioned in the -devel list recently ... but I don't know what else to do
if you-all do not want my help like this. I am *no* newbie -- I've been
involved with mainframes and later computer systems since the 1970s, always
technical (system programmer, not "applications"). So contemplate my ideas here
again very carefully, PLEASE.

Thank you.

comment:6 Changed 13 years ago by sci-fi@…

(In reply to comment #5)

(In reply to comment #4)

For the ncurses problem (unless caused by --enable, too)

Huh??? Where is --enable-ncurses anymore in MPlayer's configure script (from
svn)??? A grep of my whole local svn tree does not show 'enable-ncurses' nor
'enable-curses'. Therefore, the only way it'll enable ncurses ATPT is to detect
it.

I just saw it's named --enable-termcap or --enable-termios.

No I don't do either of these, I let ncurses be detected, wherein there's no way
to tell it to link from my newer /usr/local test version, and also it doesn't
know -ltinfo is required also. We need a better detector for those missing
symbols I showed in the first log here, probably in the form of tests for two or
more such modules. Again to be crystal clear about this: _both_ -l libs _are_
required with Apple's Tiger provided build of ncurses (what mplayer gets linked
to without my workarounds), as well as my own build (with my workarounds in
mplayer's configure script). And again I don't know why I'm the only one that's
seen this problem for the past couple years, we should ask others to verify it
with newer ncurses. It very definitely is a 'blocker' when missing symbols stop
your build.

If someone with ncurses-5.5 expertise could suggest what other module(s) we
should test for, or should we just automatically assume all 5.x versions require
both libs, I could try a couple ideas I have for a patch in this area.

(as an aside, I really wish we could rename some --enable options to match what
they _really_ look for, the semantics for 'termcap' have most definitely changed
in recent years ;) )

Thank you.

comment:7 Changed 13 years ago by reimar

Please don't try to solve all kinds of problems in one bug report, this just
makes discussion impossible. pkg-config at least in the past was quite often
broken, which is why MPlayer tries to avoid it as much as possible. But I think
this should be discussed somewhere where someone besides me will listen.
The -users list has a smaller risk of getting into flames though than the -dev
list and allows to get support from more users beforehand.
Next, I'd still like to see the configure.log "Checking for termcap" part.
On linux with ncurses 5.5 -ltinfo is definitely not needed, to either your
ncurses is different or it is actually not the termcap part that needs -ltinfo,
so adding it there will only hide the real issues.
Maybe test what happens when you use --disable-termcap.
And sorry for being a pain, but I absolutely refuse to apply fixes before I
feel that I completely understand what the issue is and what causes it.

comment:8 Changed 13 years ago by diego@…

  • Status changed from new to assigned

I mostly agree with Reimar here and the patch in its current form is rejected.
The topic should be taken to another forum, I suggest mplayer-dev-eng.

comment:9 Changed 13 years ago by sci-fi@…

Perhaps the missing ncurses symbols ought to be carried into -dev-eng for the
time being, as that might need a lot of input from various people, then we can
open a bug report to finalise it. Remember this problem initially stems from
the way Apple builds ncurses into /usr starting with Tiger IIRC, and we simply
followed suit to get the latest patches into /usr/local for testing.

Maybe the pkg-config issue could use wider discussion, too, but I'm quite
adamant about using its info "if available" over any other lib detection scheme.

In other projects such as jackit/jackd, we are already requiring a certain

level of pkg-config to be installed, and since ffmpeg/mplayer detects versions
of libs & headers, too (e.g. to enforce certain levels of live555, x264,
others), then we ought to require a known fixed version of pkg-config as well.

Let me explain why I felt compelled to use this bug reporting system. I did
mention on one of the maillists why using GMane and Mac newsreaders worked so
much better (mainly the discussion threading references are kept together
properly). But at the same time the Mac newsreaders can't create proper file
attachments that also keep the threading intact ... so that's why I am having to
use bug reports to submit patches.

After you-all read this, why don't we close this record and I'll split up the
issues into separate reports and/or post them to -dev-eng. I think we can
easily solve one of the left-over issues already, btw.

Thank you.

comment:10 Changed 13 years ago by diego@…

  • Resolution set to worksforme
  • Status changed from assigned to closed

(In reply to comment #9)

After you-all read this, why don't we close this record and I'll split up the
issues into separate reports and/or post them to -dev-eng. I think we can
easily solve one of the left-over issues already, btw.

Whatever you say.

Note: See TracTickets for help on using tickets.