[tldoc] suggestions for the doc section 3.4.5 context lmtx and mkiv
Carlos
linguafalsa at gmail.com
Fri Feb 24 19:43:27 CET 2023
On 2/24/23 08:20, Hans Hagen wrote:
> On 2/24/2023 1:52 PM, Carlos wrote:
>> On 2/23/23 17:37, Karl Berry wrote:
>>> Hi Carlos,
>>>
>>> I had some issues with the the way by which namely, context, and
>>> the
>>> scripts used therein handles any call with a file.
>>>
>>> I admit I'm not sure at what level the problem you're discussing is.
>>
>> I'm not sure either. I'm not. I thought of TeX Live documentation,
>> mainly (believe it not), after spending quite a while frustrated
>> while trying to keep up with a recurring issue, and it was through the
>> manual indeed which seemed to resolve it But the problem shouldn't
>> have been there in the first place.
>> If
>>> some specific change to the TL manual that would help, please draft
>>> something and let me know.
>>>
>>> In general, we cannot document detailed/exact ConTeXt (or anything else)
>>> behavior in the TL manual, especially wrt what envvars are used or not
>>> used, how they conflict, etc. It has to be left to the documentation of
>>> the formats/programs. If it's not documented there, we can't solve that.
>>
>> I wish I could draft just an excerpt for this. I wish. A suggestion
>> for a solution that is, that would tackle the source, and bypassing
>> envvars
>>>
>>> If you think ConTeXt should behave differently, the change needs to be
>>> sent to the ConTeXt developers. --best, karl.
>>>
>>
>> Sorry Karl. But I'm not here to tell them what they must do. I'm more
>> concerned with the tl guide than anything else, really. Don't forget
>> that it was solely by going over the tl guide by which the issue was
>> resolved in my end. I had given up with this.
>>
>> But let's go over the following.
>>
>> If you were to go over this fragment on the tl guide that says
>>
>> \dirname{TEXMFVAR} is the location for storing most cached runtime data
>> specific to each user. \dirname{TEXMFCACHE} is the variable name used
>> for that purpose by Lua\LaTeX\ and \ConTeXt\ MkIV (see
>> section~\ref{sec:context-mkiv}, \p.\pageref{sec:context-mkiv}); its
>> default value is \dirname{TEXMFSYSVAR}, or (if that's not writable),
>> \dirname{TEXMFVAR}.
>>
>>
>> Both LuaLaTeX and ConTeXt are included. Correct?
>>
>> And it clearly says that the default value for ConTeXt is TEXMFSYSVAR
>>
>> It would not be surprising at all then, to find out that LuaLaTeX
>> simply does not seem affected at all by this issue, does it? and for a
>> good reason: presumably because of the kpathsea library.
>>
>> But I genuinely can't remember if it ever happened with LuaTeX, and if
>> it did, I missed it, but with ConTeXt (whether standalone or not) this
>> is a recurring event. Give or take a handful of years, add or subtract
>> a couple of them in between. It's a cycle Karl. A cycle.
>>
>> Anyway. Here's a possible draft for the doc, and from what I've seen
>> after breaking down my own system with this very same issue.
>>
>> diff --git a/texlive-en.tex b/texlive-en.tex
>> index 6c7c572..09b2529 100644
>> --- a/home/ce/Downloads/texlive-en.tex
>> +++ b/texlive-en.tex
>> @@ -1187,6 +1187,10 @@ for LMTX, and
>> \begin{sverbatim}
>> context --luatex --generate
>> \end{sverbatim}
>> +or
>> +\begin{sverbatim}
>> +mtxrun --generate
>> +\end{sverbatim}
>> for MkIV, to refresh the \ConTeXt{} disk cache data.
>> The resulting files are stored under \code{TEXMFCACHE},
>> whose default value in \TL\ is \verb+TEXMFSYSVAR;TEXMFVAR+.
>> @@ -1196,6 +1200,18 @@ all paths mentioned in \verb+TEXMFCACHE+, and
>> write to the first
>> path that is writable. While reading, the last found match will
>> take precedence in the case of duplicated cache data.
>>
>> +It is recommended for the stored cache data on \ConTeXt\ to be
>> +written under \code{TEXMFVAR} preferably over \code{TEXMFSYSVAR}.
>> +
>> +Duplicate cache data on both \code{TEXMFVAR} and \code{TEXMFSYSVAR}
>> +could create a conflict while generating and updating the formats. This
>> +may result in an error such as:
>> +
>> +\verb+mtxrun | unknown script `context.lua' or `mtx-context.lua'+
>> +
>> +The \dirname{luatex-cache} directories under both \code{TEXMFVAR} and
>> +\code{TEXMFSYSVAR} would normally hold the duplicate runtime data cache.
>> +
>> For more information, see \url{https://wiki.contextgarden.net/LMTX}\\
>> and \url{https://wiki.contextgarden.net/Running_Mark_IV}.
>
> fwiw:
>
> the caching code hasn't changed in over a decade; normally context
> itself will generate the file database and format (when it notices a
> change in engine or context version)
>
> the context cache paths are hashed in a way that also stores the path so
> conflicts are unlikely; there are dedicated subpaths for 'luatex' and
> 'luametatex' and even dedicated suffixes in some cases (luajittex is
> still supported for mkiv but eventually it might get dropped)
>
> when lmtx / luametatex is installed, the 'runner' is actually luametatex
> the choice of the engine --luatex flag to the 'context' runner (which is
> a copy of or symlink to luametatex as is mtxrun)
>
> in an editor one can configure
>
> mtxrun --autogenerate --script context <filename>
>
> which will generate the file database and format if needed (and use it
> in runs after that)
>
> so, there are three binaries (two are actually stubs, so the old runner
> - runscript or so, i'm not sure what was used in texllive - is now
> obsolete, which is actually good because that one occasionally got
> adapted out of our sight)
>
> luametatex
> mtxrun -> luametatex
> context -> luametatex
>
> and
>
> two lua files (in the bin path):
>
> mtxrun.lua (also resolves --script specs)
> context.lua (eventually loads mtx-context.lua)
>
> it's the same on all platforms.
>
> the only complication i can imagine is that when the bin path is not
> relative to the rest the auto* stuff might not work as expected but
> there's not much i can do about that (it's then also controlled by env
> vars)
>
> Hans
>
> -----------------------------------------------------------------
> Hans Hagen | PRAGMA ADE
> Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
> tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
> -----------------------------------------------------------------
>
>
You're right. 'Conflict' may have been the wrong term here.
There are two (2) issues here.
The first issue which is unsettling is having context trying to sneak
in and read even read-protected files. Just as a `find' utility would do.
The second issue is that very same error message of `mtxrun | unknown
scrpit 'context.lua' or 'mtx-context.lua' as above.
If I invoke kpsewhich --var-value=TEXMFCACHE it returns:
/usr/share/texmf-var:/.texlive2022/texmf-var
But then again, if I were to run `context --autogenerate --script
context <filename>` it fully stalls right before coming across a
write-read-protected dir.
If on the other hand I were to run `mtxrun --script cache --erase`
this would fetch the results of only one (1) hashed dir, namely the
one with the fonts, formats, and trees.
mtxrun --script cache --erase
mtx-cache | writable path:
/.texlive2022/texmf-var/luatex-cache/context/0399a8df3aef8d154781d0a9c2b8e28d
mtx-cache |
mtx-cache | fmt : 1
mtx-cache | log : 1
mtx-cache | lua : 4
mtx-cache | luc : 4
mtx-cache | lui : 1
mtx-cache | luv : 1
mtx-cache | tma : 2
mtx-cache | tmc : 2
mtx-cache |
mtx-cache | total : 16
mtx-cache |
mtx-cache | writable path:
/.texlive2022/texmf-var/luatex-cache/context/0399a8df3aef8d154781d0a9c2b8e28d
mtx-cache |
mtx-cache | fmt : 1 gone, 0 kept
mtx-cache | log : 1 gone, 0 kept
mtx-cache | lua : 4 gone, 0 kept
mtx-cache | luc : 4 gone, 0 kept
mtx-cache | lui : 1 gone, 0 kept
mtx-cache | luv : 1 gone, 0 kept
mtx-cache | tma : 2 gone, 0 kept
mtx-cache | tmc : 2 gone, 0 kept
mtx-cache |
The other hash is simply ignored. Apparently because the fonts dir is
not part of it, or simply because it was another unrelated context
database generate instance.
Hans, I appreciate your reply. With the utmost respect to you and the
legacy of your work behind it.
But In the current state of affairs, as the saying goes, as of this
writing, and to put it mildly
In order to run and generate a database for ConTeXt, a new directory
owned by the user must be created. Or else, no hashed dirs will be
created, no fonts will be loaded, and obviously the formats will not
even get updated.
Likewise, if any directory is not owned by the user from where any
future `context` invocation would be called out, any subsequent
`context --generate` or `mtxrun --generate` will be altogether ignored.
I guess that the above ought to be looked at, from a caveat emptor
perspective.
I don't say this in a demeaning way. I can see the rationale behind it,
but with the same token, it's just discomforting
So even after going through all of this:
resolvers | resolving | locating list of '/usr/share/texmf-dist'
(cached)
resolvers | methods | resolving, method 'locators', how 'uri',
handler 'file', argument '/usr/share/texmf-dist'
resolvers | files | file locator '/usr/share/texmf-dist' found as
'/usr/share/texmf-dist'
resolvers | resolving |
resolvers | methods | resolving, method 'generators', how 'uri',
handler 'file', argument '/usr/share/texmf-config'
resolvers | expansions | using cached scan of path
'/usr/share/texmf-config', branch '/usr/share/texmf-config'
resolvers | methods | resolving, method 'generators', how 'uri',
handler 'file', argument '/usr/share/texmf'
resolvers | expansions | using cached scan of path
'/usr/share/texmf', branch '/usr/share/texmf'
resolvers | methods | resolving, method 'generators', how 'uri',
handler 'file', argument '/usr/share/texmf-config'
resolvers | expansions | using cached scan of path
'/usr/share/texmf-config', branch '/usr/share/texmf-config'
resolvers | methods | resolving, method 'generators', how 'uri',
handler 'file', argument '/usr/share/texmf-var'
resolvers | expansions | scanning path '/usr/share/texmf-var',
branch '/usr/share/texmf-var'
resolvers | expansions | files found on directories with 1
uppercase remappings
resolvers | methods | resolving, method 'generators', how 'uri',
handler 'file', argument '.'
resolvers | expansions | scanning path '.', branch '.'
Any further subdirectory created within it (that is, within the same
directory on which both mtxrun | unknown script 'context.lua' or
'mtx-context.lua' could not have been found/loaded before, would
henceforth get all read and write privileges.
take care Hans. Karl… see you around.
More information about the tldoc
mailing list.