path issue when using phar

[arabcoders]

It seems there is problem with loading Phar files when using zend opcache, i trace it down to path issues.

for example when you use the following script in Mutliple Hosts Environment (MHE), it does not work and the autoload does not find and scripts, and my guess is that the opcache is loading the other websites phar and they have different owner & permmsions so its fails to load them.

<?php
$baseDir = './';
include("phar://{$baseDir}include/twig.phar/Twig/Autoloader.php");
Twig_Autoloader::register();
$loader = new Twig_Loader_Filesystem($baseDir . '/tpl/');

?>

but changing the baseDir to

<?php
$baseDir = '/home/user/public_html/';
include("phar://{$baseDir}include/twig.phar/Twig/Autoloader.php");
$loader = new Twig_Loader_Filesystem($baseDir . '/tpl/');
?>

it seems to work alright, and Yes i tried the first script with opcache disabled and it worked

the same problem happens with APC as well.

[dstogov]

The first script works fine for me with OPcache.
However, I'm not sure what do you ,mean by MHE. Probably, it is the real reason of the problem, but I don't know how to reproduce it.

Anyway, it must be more robust to substitute './' with DIR constant.

[arabcoders]

sorry i didn't explain the problem correctly,

I run a small web hosting server for friends and they have 100's of websites using PHP 5.5 with OPcache cause problems with scripts that use relative paths IE (./../) for example lets say most of them use wordPress or zend framework and someone made awesome plugin that is contained in phar file, if you use relative paths it somehow triggers a bug in OPcache that try to load the phar not from the website that requested the file but from the other website that has the same plugin.

i hope i made it clear enough for you to understand what i mean

PS: English is not my native tongue

[dstogov]

I understood, I'm not a native speaker as well :)

Anyway, to fix the problem, I need some way to reproduce it.
If you may provide a detailed instruction it would be helpful.

Thanks. Dmitry.

On Thu, Aug 1, 2013 at 5:30 AM, Abdul.Mohsen B. A. A. <
notifications@github.com> wrote:

sorry i didn't explain the problem correctly,

I run a small web hosting server for friends and they have 100's of
websites using PHP 5.5 with OPcache cause problems with scripts that use
relative paths IE (./../) for example lets say most of them use wordPress
or zend framework and someone made awesome plugin that is contained in phar
file, if you use relative paths it somehow triggers a bug in OPcache that
try to load the phar not from the website that requested the file but from
the other website that has the same plugin.

i hope i made it clear enough for you to understand what i mean

PS: English is not my native tongue

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/115#issuecomment-21908549
.

[arabcoders]

ok i'll try to make testcase tomorrow once i get back to my workstation

[travispaul]

Does ZendOptimizerPlus support Phars in general? I thought it would be better to ask here than open a new issue.

[dstogov]

It must support phar. At least OPCache passes all the tests from ext/phar/tests.

[travispaul]

Thanks @dstogov

[pilif]

I'm running into this, or a similar issue. I'll try to produce a really reduced testcase which I'll append here, but let me give you an overview.

I have php-fpm running, serving two different versions of an application from two different directories.

Both application use a phar-file (which gets include()d directly) with the same name, but with different contents (the two applications are the same, but at different releases, so one phar file is newer than the other and between the two, some reorganization has happened).

Inside the phar file are multiple files which are included as part of the phar's stub.

Now the phar file itself is cached correctly, but their contents isn't.

Whichever phar file is loaded first has its contents stored in the cache which is then used to serve all future requests for files from an equally named phar file (but with different content).

[pilif]

ok. I can offer a reduced testcase now, though as it involves multiple vhosts, it's still a bit painful to set up. Sorry for that 😦

First, opcache is configured with

   opcache.use_cwd=1
   opcache.revalidate_path=1
   opcache.validate_timestamps=1
   opcache.revalidate_freq=0

So this is as "refreshy" as it gets.

Next, configure two vhosts using the files I've configured in this archive. Point each one to either the newer or older subdir.

The phar file in question is a php file in disguise (block.asset_compile.php - it's form my own sacy project which is a smarty plugin to compress assets. Both index.phps in both vhosts include the file and then run the plugin function with some mock data.

Now with a freshly restarted php-fpm, the first vhost you hit will work correctly, but the second one you hit will complain about a class not being found (it's either sacy_Config if you run the vhost named "newer" first, or sacy\Config if you run "older" first.

If you restart fpm and hit the respective other vhost first, the first hit, again, will work correctly, but the second one will fail.

As I said in my previous comment, I believe that opcache doesn't correctly namespace the files inside of the phar file, so the following statements from the phar stub:

include("phar://sacy.phar/sacy/ext-translators.php");
include("phar://sacy.phar/sacy/sacy.php");

will immediately hit the cached variant, despite the actual phar file being located at different locations on the file system.

Thinking about this, I might even be able to reduce this testcase even further, brining this down to one vhost. If I can, I'll post another comment here.

[pilif]

Yes. My previous suspicions were correct - this doesn't require two vhosts at all.

If you use this archive (unpack to DOCUMENT_ROOT - I'm using a hard-coded include_path to show that it's not an include_path related issue) and you restart php-fpm, then you can either run using_older.php or using_newer.php, but whichever you run first is the only one you'll be able to run.

When you restart fpm (to flush the cache), then you can switch the file. The first one will always be fine, the second one will always fail.

So I stand by my initial investigation: opcache seems to take the symbolic name (using Phar::mapPhar) inside the phar as a cache key without taking the actual phar file's location into account.

[dstogov]

I suspect OPCache can't make difference between Phar aliases created using Phar::mapPhar(),
and I don't see a way to solve it.
To workaround the problem it's probably possible to use a full path name to current file.
e.g.

$phar = __FILE__ . "/sacy.phar";
Phar::mapPhar($phar);
include("phar://$phar/sacy/ext-translators.php");
include("phar://$phar/sacy/sacy.php");

[pilif]

Yeah. I agree. The thing is though: In plain PHP, this works and so it does in 5.4 and earlier with APC, so this must at least be documented somewhere, or, considering that this is kinda dangerous, opcache should opt out of caching/optimizing stuff inside a phar archive.

In shared hosting environments, this is going to bite somebody I'm sure.

[TerryE]

IMHO, both Phar and OPcache are core components of PHP, so these two extensions should interoperate seamlessly; virtualization of the phar root is a core feature that enables abstraction of any phar'ed application so I don't feel that the sort of workaround suggested by Dmitry is the right approach.

I generated a phar from the following PHP stub in /tmp and executed there as well

<?php
    # phar.readonly must be 0
    $stub = '<?php
    Phar::interceptFileFuncs();
    set_include_path("phar://this" . PATH_SEPARATOR . get_include_path());
    require "index.php";
    __HALT_COMPILER(); ?>';

    $p = new Phar(dirname(__FILE__) . '/newphar.phar.php', 0, 'this');
    $p['index.php'] = '<?php echo "index is brand new!\n"; require "hello.php";require "hello.php";';
    $p['hello.php'] = '<?php echo "and hello from me!\n";';    
    $p->setStub($stub);
    unset($p);

I traced through the execution of both current PHP 5.4 and 5.5 versions with gdb. Both executions are functionally equivalent. At accel_deactivate() the ZCG(include_paths) table contains two entries ['A' => '.', 'B' => 'phar://this:.'] and the ZCG(hash) table contains three direct entries for:

• /tmp/newphar.phar.php
• phar:///tmp/newphar.phar.php/index.php
• phar:///tmp/newphar.phar.php/hello.php

So for so good, but it also includes indirect entries for:

• /tmp:index.php:/tmp:B
• /tmp:hello.php:phar:///tmp/newphar.phar.php:B

Hence subsequent fetches of both index.php and hello.php will match against the indirect keys entries (which map through the include_path onto the alias), with the entries pointing to the respective fully resolved entries. The issue comes when another phar is executed, say /tmp/otherphar.phar.php, and which also establishes an alias for this. The indirect entries will still be pointing the newphar entries, hence this bug.

The safest way of avoiding this would be for accel_make_persistent_key_ex() to explicitly resolve any phar URIs to convert any phar alias into its fully resolved absolute form. As far as I can see, there isn't a phar C callable entry to resolve a named alias within the current request (but I am not a phar expert); this is what we need to implement this. I could do some more investigation, but this is one that the phar maintainers should make know how to make quickly available.

If someone from the phar developers can suggest how to do this alias resolution, I can do the OPcache changes.

[TerryE]

I've done some enumeration around use cases looking at OPcache internals. The issue is wider than I've described above. At its root, phar embeds the concept of aliases which can symbolically map names onto file system paths in the same way as file system symlinks do. The issue for OPcache is that aliases are scoped to the request and therefore consecutive requests can map any alias, e.g. this (as in the above example), to different paths. On the other hand OPcache assumes that the key :: is an invariant for the life of the cache.

The only safe way to address this is to assume that any phar:/// might include such an alias and attempt to resolve it. At the moment the Phar extension doesn't provide this API function. It's only a few lines of C do this but this assumes access to Phar internal data structures. Better to add a C callable external (a dozen lines of C to be added to the Phar source) to some externally exposed header. Anyone including me code do this PHar code, but again this requires the endorsement of the Phar maintainers.

@dstogov, Dmitry you know who to pass this to. Can you please forward. Thanks.

[dstogov]

you are exactly right about phar aliases. OPCache needs a way to resolve
them into full realpath name of the phar file,

On Mon, Oct 14, 2013 at 10:58 PM, Terry Ellison notifications@github.comwrote:

I've done some enumeration around use cases looking at OPcache internals.
The issue is wider than I've described above. At its root, phar embeds the
concept of aliases which can symbolically map names onto file system
paths in the same way as file system symlinks do. The issue for OPcache is
that aliases are scoped to the request and therefore consecutive requests
can map any alias, e.g. this (as in the above example), to different
paths. On the other hand OPcache assume that the key :: is an invariant for
the life of the cache.

The only safe way to address this is to assume that any phar:/// might
include such an alias and attempt to resolve it. At the moment the Phar
extension doesn't provide this API function. It's only a few lines of C do
this but this assumes access to Phar internal data structures. Better to
add a C callable external (a dozen lines of C to be added to the Phar
source) to some externally exposed header. Anyone including me code do this
PHar code, but again this requires the endorsement of the Phar maintainers.

@dstogov https://github.com/dstogov, Dmitry you know who to pass this
to. Can you please forward. Thanks.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/115#issuecomment-26279751
.