Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Illegal State issue from initdb #65

Closed
jweaver-pfi opened this issue Feb 6, 2018 · 44 comments
Closed

Illegal State issue from initdb #65

jweaver-pfi opened this issue Feb 6, 2018 · 44 comments

Comments

@jweaver-pfi
Copy link

Hi, I'm attempting to use otj-pg-embedded-0.11.0 (as downloaded from the maven repository) on a windows 10 system and I am getting the following when I attempt a simple junit test:

java.lang.IllegalStateException: Process [C:\Users\john\AppData\Local\Temp\embedded-pg\PG-20610842f7206a9abbecdcf12709c0f8\bin\initdb.exe, -A, trust, -U, postgres, -D, C:\Users\john\AppData\Local\Temp\epg3467157523703904606, -E, UTF-8] failed

at com.opentable.db.postgres.embedded.EmbeddedPostgres.system(EmbeddedPostgres.java:558)
at com.opentable.db.postgres.embedded.EmbeddedPostgres.initdb(EmbeddedPostgres.java:223)
at com.opentable.db.postgres.embedded.EmbeddedPostgres.<init>(EmbeddedPostgres.java:141)
at com.opentable.db.postgres.embedded.EmbeddedPostgres$Builder.start(EmbeddedPostgres.java:545)
at com.opentable.db.postgres.embedded.EmbeddedPostgres.start(EmbeddedPostgres.java:445)
at com.opentable.db.postgres.junit.SingleInstancePostgresRule.before(SingleInstancePostgresRule.java:35)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:41)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

The unit test is copied from from your test:

package com.billerica.manager.database;

import com.opentable.db.postgres.junit.EmbeddedPostgresRules;
import com.opentable.db.postgres.junit.SingleInstancePostgresRule;
import org.junit.Rule;
import org.junit.Test;

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.Statement;

import static org.junit.Assert.*;

public class testSingleInstanceEmbeddedPg {

@Rule
public SingleInstancePostgresRule epg = EmbeddedPostgresRules.singleInstance();

@Test
public void testSingleInstance() throws Exception {
    try (Connection c = epg.getEmbeddedPostgres().getPostgresDatabase().getConnection()) {
        assertNotNull(epg);
        Statement s = c.createStatement();
        ResultSet rs = s.executeQuery("SELECT 1");
        assertTrue(rs.next());
        assertEquals(1, rs.getInt(1));
        assertFalse(rs.next());
    }
}

}

I see that the temp folder epg3467157523703904606 is empty, if that says anything. I haven't found any log files in the temporary folders.

Other tests give similar results. Sorry if I have missed something obvious; I haven't done any other configuration or setup.

Thanks for any help you can provide.

@stevenschlansker
Copy link
Contributor

Did postgres print anything to standard error before failing? initdb should have emit some output and you need to see what that says to understand why it's failing.

@jweaver-pfi
Copy link
Author

Hi Steven, thanks for the reply.

I've been unable to find any sign of logging other than the stack trace (I was running from an IntelliJ IDEA terminal)

However, quite strangely the tests are now passing successfully. So something in my environment has changed to make it work. Previously I certainly had tried a number a times (including shutting down IDEA and rebuilding from scratch.

Something I did do in the meantime, was that I did install Postgres (and drivers for it) on the windows box. Could that have made the difference? My understanding of otj-pg-embedded was that it comes complete with a bundled postgres. Was I wrong in my understanding?

In any case, since things are now functioning (and I can no longer reproduce the problem) I guess we can close this ticket out. I would be interested in any comments you might have on what could have caused this, although there is very little to go on. Sorry for your time.

@volphy
Copy link

volphy commented Feb 19, 2018

I have the same issue locally.

My environment:

  • MS Windows 10 (build 1607 with all the updates)
  • JDK: Oracle HotSpot 1.8.0_161
  • Apache Maven: 3.5.2 (the issue also occurs when running the test without Maven i.e. directly from IDE)
  • IDE: IntelliJ IDEA Ultimate IntelliJ IDEA 2017.3.4 (Ultimate Edition) Build #IU-173.4548.28, built on January 29, 2018
  • Postgresql-embedded: 2.6

Neither setting all logs to DEBUG/TRACE level or debugging helped me.

Running everything as administrator does not change anything.

Any idea?

@volphy
Copy link

volphy commented Feb 22, 2018

I have observed this issue while using otj-pg-embedded 0.11.0.

Unfortunately, it still occurs while running 0.11.1 release.

@stevenschlansker
Any idea?

@mikebell90
Copy link
Contributor

Since none of us at OT currently work with Windows, we are happy to entertain PR's, but can't really resolve it (assuming it's windows specific)

What would be really great is if one of you would run initdb manually on a new postgres install

@volphy
Copy link

volphy commented Mar 2, 2018

I do not run PostgreSQL as a separate Windows process.

In this scenario I used Embedded PostgreSQL:
https://github.com/yandex-qatools/postgresql-embedded

Embedded PostgreSQL 2.6 (latest release at the time of writing) runs PostgreSQL 9.6.7 in-memory.

Running initdb.exe manually provides no output:
`
D:\git>C:\Users\krzwil\AppData\Local\Temp\embedded-pg\PG-20610842f7206a9abbecdcf12709c0f8\bin\initdb.exe -A, trust -U postgres -D C:\Users\krzwil\AppData\Local\Temp\epg4200883402029108528 -E UTF-8

`

Is there a way to enforce debug/trace messages on initdb.exe?

@jweaver-pfi
Copy link
Author

Just as a quick update. I have not seen a recurrence of this issue, and cannot reproduce it. Also, a colleague here at work (who did not have PostgreSQL previously installed, and is running with a similar environment to mine) also does not hit this problem while running tests using embedded postgres.

@volphy
Copy link

volphy commented Mar 5, 2018

I have tried to more things:

  • removing files from C:\Users\krzwil\AppData\Local\Temp\ before running tests
  • upgrading to postgresql-embedded from 2.6 to 2.8

None of them changed reported issue a bit.

Any idea?

@volphy
Copy link

volphy commented Mar 5, 2018

I have also tried running all tests ('mvn test') on freshly cloned postgresql-embedded master branch.

They failed:
`
T E S T S

Running ru.yandex.qatools.embed.postgresql.config.StorageTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.443 sec
Running ru.yandex.qatools.embed.postgresql.EmbeddedPostgresTest
Extract C:\Users\krzwil.embedpostgresql\postgresql-10.2-1-windows-x64-binaries.zip START
...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Extract C:\Users\krzwil.embedpostgresql\postgresql-10.2-1-windows-x64-binaries.zip DONE
2018-03-05 10:15:37 DEBUG ProcessControl:117 - Detected pid: 11616
2018-03-05 10:15:52 INFO Executable:109 - start AbstractPostgresConfig{storage=Storage{dbDir=C:\Users\krzwil\AppData\Local\Temp\postgresql-embed-4a5ac605-90cc-4596-9591-ecfc6bd8a629\db-content-d5265410-a6dc-4c47-a6c4-ef34e960f92b, dbName='postgres', isTmpDir=true}, network=Net{host='localhost', port=3725}, timeout=Timeout{startupTimeout=15000}, credentials=Credentials{username='postgres', password='postgres'}, args=[], additionalInitDbParams=[-E, SQL_ASCII, --locale=C, --lc-collate=C, --lc-ctype=C]}
2018-03-05 10:15:54 WARN PostgresProcess:122 - Possibly failed to run initdb:

2018-03-05 10:15:54 DEBUG ProcessControl:117 - Detected pid: 12292
2018-03-05 10:16:14 ERROR PostgresProcess:257 - Failed to read PID file (File 'C:\Users\krzwil\AppData\Local\Temp\postgresql-embed-4a5ac605-90cc-4596-9591-ecfc6bd8a629\db-content-d5265410-a6dc-4c47-a6c4-ef34e960f92b\postmaster.pid' does not exist)
java.io.FileNotFoundException: File 'C:\Users\krzwil\AppData\Local\Temp\postgresql-embed-4a5ac605-90cc-4596-9591-ecfc6bd8a629\db-content-d5265410-a6dc-4c47-a6c4-ef34e960f92b\postmaster.pid' does not exist
at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:299)[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.readLines(FileUtils.java:1783)[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.readLines(FileUtils.java:1819)[commons-io-2.4.jar:2.4]
at ru.yandex.qatools.embed.postgresql.PostgresProcess.onAfterProcessStart(PostgresProcess.java:255)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at de.flapdoodle.embed.process.runtime.AbstractProcess.(AbstractProcess.java:116)[de.flapdoodle.embed.process-2.0.2.jar:]
at ru.yandex.qatools.embed.postgresql.AbstractPGProcess.(AbstractPGProcess.java:19)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at ru.yandex.qatools.embed.postgresql.PostgresProcess.(PostgresProcess.java:68)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at ru.yandex.qatools.embed.postgresql.PostgresExecutable.start(PostgresExecutable.java:25)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at ru.yandex.qatools.embed.postgresql.PostgresExecutable.start(PostgresExecutable.java:13)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at de.flapdoodle.embed.process.runtime.Executable.start(Executable.java:108)[de.flapdoodle.embed.process-2.0.2.jar:]
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:173)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:142)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:134)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:130)[file:/D:/tmp/postgresql-embedded/target/classes/:]
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgresTest.itShouldStartWithDefaults(EmbeddedPostgresTest.java:41)[file:/D:/tmp/postgresql-embedded/target/test-classes/:]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.8.0_161]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:1.8.0_161]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_161]
at java.lang.reflect.Method.invoke(Method.java:498)[:1.8.0_161]
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)[junit-4.12.jar:4.12]
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)[junit-4.12.jar:4.12]
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)[junit-4.12.jar:4.12]
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)[junit-4.12.jar:4.12]
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)[junit-4.12.jar:4.12]
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)[junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)[junit-4.12.jar:4.12]
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)[junit-4.12.jar:4.12]
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)[junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)[junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)[junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)[junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)[junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)[junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)[junit-4.12.jar:4.12]
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.8.0_161]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:1.8.0_161]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_161]
at java.lang.reflect.Method.invoke(Method.java:498)[:1.8.0_161]
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)[surefire-api-2.12.4.jar:2.12.4]
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)[surefire-booter-2.12.4.jar:2.12.4]
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)[surefire-booter-2.12.4.jar:2.12.4]
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)[surefire-booter-2.12.4.jar:2.12.4]
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)[surefire-booter-2.12.4.jar:2.12.4]
`

Is it an otj-pg-embedded issue or postgresql-embedded issue then?

@alexanderjamesking
Copy link

I can replicate this one too:

(test-system/start-system!) 18-03-21 16:24:56 mll1p05653.local INFO [com.opentable.db.postgres.embedded.EmbeddedPostgres:668] - Detected a Darwin x86_64 system 18-03-21 16:24:56 mll1p05653.local INFO [com.opentable.db.postgres.embedded.EmbeddedPostgres:707] - Extracting Postgres... 18-03-21 16:25:01 mll1p05653.local INFO [com.opentable.db.postgres.embedded.EmbeddedPostgres:742] - Postgres binaries at /var/folders/_s/zh8x143s1dbf2m0b076b9st1b7y13m/T/embedded-pg/PG-1576ace94f039fde0d83e95305e0acc5 IllegalStateException Process [/var/folders/_s/zh8x143s1dbf2m0b076b9st1b7y13m/T/embedded-pg/PG-1576ace94f039fde0d83e95305e0acc5/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/_s/zh8x143s1dbf2m0b076b9st1b7y13m/T/epg570193259857342578, -E, UTF-8] failed com.opentable.db.postgres.embedded.EmbeddedPostgres.system (EmbeddedPostgres.java:558) user>

@volphy
Copy link

volphy commented Mar 22, 2018

@alexanderjamesking thanks for confirming that someone else observes the same issues as me

To sum up, I have checked the following environments:

  • MS Windows 10, version 1607 (w/ corporate policies running on a business laptop) - FAILS
  • MS Windows 10, version 1703 (vanilla running in VMware machine on a private laptop) - WORKS
  • MS Windows 10, version 1709 (vanilla running in VMware machine on a private laptop) - WORKS
  • Maven as Docker container https://hub.docker.com/_/maven/ (root user inside Docker container) - FAILS

@byegor
Copy link

byegor commented Mar 26, 2018

hi guys, i had the same issue, but when i run initdb.exe manually - widows alert me that some dll is not found, so just download it and it should help

Cheers

@volphy
Copy link

volphy commented Apr 19, 2018

Thanks @Egoruch for your piece of advice.

I have installed MS Visual C++ Redistributable Packages for Visual Studio 2013 x64 and it helped me.

I had already installed MS Visual C++ Redistributable Packages for Visual Studio 2013 x86 but it turned out not to be sufficient.

@mikebell90
Copy link
Contributor

Makes sense! Thanks! We will add to readme then close this

@mikebell90
Copy link
Contributor

Closing, since the README is now updated.

@kartoffelsack
Copy link

Had the same issue, even after I had installed Microsoft Visual C++ Redistributable for Visual Studio 2017 (64 bit).

What helped was to install MS Visual C++ Redistributable Packages for Visual Studio 2013 x64 :-)

Thanks!

@mikebell90
Copy link
Contributor

Right - these are static - and depend on what Postgres was compiled on at the time

@Hawk707
Copy link

Hawk707 commented Jul 8, 2018

I had different error message. I had run time exception stating that inistdb.exe was not found under \user\AppData\Local\Temp\embedded-pg\PG-20610842f7206a9abbecdcf12709c0f8\bin\

This happened without any changes from except I added extra test, which threw the above exception. Then all other tests, which used to work just fine failed. Even another Application on the same machine threw the same exception.

I downloaded MS Visual C++ Redistributable Packages for Visual Studio 2013 x64. I did not help. I was using 0.11.0. I upgraded to 0.12.0, and they all worked. Not sure what happened all of a sudden!

@tuliogrodrigues
Copy link

tuliogrodrigues commented Jan 8, 2019

I got the same error in a Mac machine, but seems that the problems was the user's permissions. I've tried execute the command manually and I got an error complaining about this.

command executed:

/var/folders/qc/q9c34bpj3k95fx2vfm6c7jx80000gp/T/embedded-pg/PG-d6f2320465793366f452337868c0d06f/bin/initdb -A trust -U postgres -D /var/folders/qc/q9c34bpj3k95fx2vfm6c7jx80000gp/T/epg8438268694472155640 -E UTF-8

@varunrocks6
Copy link

After debugging, and running the initdb command directly on the docker agent:

initdb: encoding mismatch The encoding you selected (UTF8) and the encoding that the selected locale uses (LATIN1) do not match.

This means that the locale wasn't set when embedded-postgres db was created. For me it worked by setting the locale:

export LANG=en_US.UTF-8

@salluspeaks
Copy link

I faced this issue on mac and seems like it was a permission issue on intellij not being able to run commands. I was successfully able to run test case from terminal using gradle.

@danielcompton
Copy link
Contributor

I've run into this issue on a Mac too. I tried removing various temp files, but I couldn't make it start correctly. Running the commands directly in my terminal was fine, but not from the JVM. After restarting the computer it ran fine. Not a satisfying result obviously, I'll do some more digging the next time this issue comes up.

@nieldw
Copy link

nieldw commented Jul 22, 2019

Running into this on Mac too. Similar to @danielcompton . Running my test with the IntelliJ runner I get one more clue:

INFO  init-190fff25-b380-41d7-af41-01fa8364adeb:initdb - The files belonging to this database system will be owned by user "myuser". [# 17:26:28.193] [log:pid(36102)]
INFO  init-190fff25-b380-41d7-af41-01fa8364adeb:initdb - This user must also own the server process. [# 17:26:28.193] [log:pid(36102)]

@kennyjwilli
Copy link

I started receiving this exception after switching images on our CI to adopopenjdk 11. After running the initdb command from inside the container, I received this output:

/tmp/embedded-pg/PG-06e3a92a2edb6ddd6dbdf5602d0252ca/bin/initdb -A trust -U postgres -D /tmp/epg7204157023524008314 -E UTF-8
initdb: cannot be run as root
Please log in (using, e.g., "su") as the (unprivileged) user that will
own the server process.

Switching the container user from root to circleci fixed this.

@svengelhardt
Copy link

I had the same problem and solved it. The exception was the following:

java.lang.IllegalStateException: Process [C:\Users\***\AppData\Local\Temp\embedded-pg\PG-82d8fdc4b690082b3ac0667aee5358bb\bin\initdb.exe, -A, trust, -U, postgres, -D, C:\Users\***\AppData\Local\Temp\epg6545158044000690790, -E, UTF-8] failed

From my spring test the following internally called command failed:

C:\Users\***\AppData\Local\Temp\embedded-pg\PG-82d8fdc4b690082b3ac0667aee5358bb\bin\initdb.exe -A trust -U postgres -D C:\Users\***\AppData\Local\Temp\epg6545158044000690790 -E UTF-8

Doing this out of the windows console worked properly (after adding the C:\Users\***\AppData\Local\Temp\embedded-pg\PG-82d8fdc4b690082b3ac0667aee5358bb\lib folder to the PATH environment variable to prevent the .dll message) and everything was created in C:\Users\***\AppData\Local\Temp\epg6545158044000690790.

Restarting the whole PC and cleaning the C:\Users\***\AppData\Local\Temp folder solved
the problem.

@sobvan
Copy link

sobvan commented Dec 20, 2019

I had the same problem on a Mac, but I was able to solve quite simply.

10:01:20.447 [DEBUG] [TestEventLogger] nyomio.commons.EmbeddedPostgresTest > testEmbedded() STANDARD_OUT
10:01:20.447 [DEBUG] [TestEventLogger]     10:01:20.442 [Test worker] INFO com.opentable.db.postgres.embedded.EmbeddedPostgres - Detected a Darwin x86_64 system
10:01:20.557 [DEBUG] [TestEventLogger]     10:01:20.557 [Test worker] INFO com.opentable.db.postgres.embedded.EmbeddedPostgres - Postgres binaries at /var/folders/4q/***/T/embedded-pg/PG-73dc0043fe7bdb624d5e8726bc457b7e
10:01:20.558 [DEBUG] [TestEventLogger]     10:01:20.558 [Test worker] DEBUG com.opentable.db.postgres.embedded.EmbeddedPostgres - 788f627d-e723-40a9-980a-f7b202965bb7 postgres: data directory is /var/folders/4q/***/T/epg7982076833040837593, postgres directory is /var/folders/4q/***/T/embedded-pg/PG-73dc0043fe7bdb624d5e8726bc457b7e
10:01:20.650 [DEBUG] [TestEventLogger]     10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - The files belonging to this database system will be owned by user "istvan".
10:01:20.650 [DEBUG] [TestEventLogger]     10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - This user must also own the server process.
10:01:20.650 [DEBUG] [TestEventLogger]     10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - 
10:01:20.650 [DEBUG] [TestEventLogger]     10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - initdb: invalid locale settings; check LANG and LC_* environment variables
10:01:20.671 [DEBUG] [TestEventLogger] 
10:01:20.671 [DEBUG] [TestEventLogger] nyomio.commons.EmbeddedPostgresTest > testEmbedded() FAILED
10:01:20.672 [DEBUG] [TestEventLogger]     java.lang.IllegalStateException: Process [/var/folders/4q/zq9fy2hs1kn3xcyt333x_l240000gn/T/embedded-pg/PG-***/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/4q/***/T/epg7982076833040837593, -E, UTF-8] failed
10:01:20.672 [DEBUG] [TestEventLogger]         at com.opentable.db.postgres.embedded.EmbeddedPostgres.system(EmbeddedPostgres.java:602)
10:01:20.673 [DEBUG] [TestEventLogger]         at com.opentable.db.postgres.embedded.EmbeddedPostgres.initdb(EmbeddedPostgres.java:221)
10:01:20.673 [DEBUG] [TestEventLogger]         at com.opentable.db.postgres.embedded.EmbeddedPostgres.<init>(EmbeddedPostgres.java:142)
10:01:20.673 [DEBUG] [TestEventLogger]         at com.opentable.db.postgres.embedded.EmbeddedPostgres$Builder.start(EmbeddedPostgres.java:554)
10:01:20.673 [DEBUG] [TestEventLogger]         at com.opentable.db.postgres.embedded.EmbeddedPostgres.start(EmbeddedPostgres.java:438)
10:01:20.673 [DEBUG] [TestEventLogger]         at nyomio.commons.EmbeddedPostgresTest.testEmbedded(EmbeddedPostgresTest.kt:13)

The line initdb: invalid locale settings; check LANG and LC_* environment was the hint.

You do have to set your localization environment variables to UTF8 like this:

export LC_ALL="en_US.UTF-8"
export LC_CTYPE="en_US.UTF-8"

(according to this stackoverflow answer.)

Of course, you can set these environment variables in your intelliJ runtime config.

Or even better, if you use gradle, you can add environment variables to the test task like this:

test {
    useJUnitPlatform()
    environment "LC_CTYPE", "en_US.UTF-8"
    environment "LC_ALL", "en_US.UTF-8"
}

I cannot try this on windows, but it might help there, too. As far as I know, on windows you have to set the LANG environment variable.

@wtangv
Copy link

wtangv commented Feb 16, 2020

Here like to share my problem and solution.
1> On Mac, using otj-pg-embedded-0.13.1 and 0.13.3. It worked before, but not any more (being long time, there were many different updates onto my Mac).
2> Errors (same from shell command line and Intelij):
java.lang.IllegalStateException:
Process [/var/folders/lj/vg60qx992j5frpmgbs498w940000gp/T/embedded-pg/PG-73dc0043fe7bdb624d5e8726bc457b7e/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/lj/vg60qx992j5frpmgbs498w940000gp/T/epg1897529519870188961, -E, UTF-8] failed
3> Tried suggestions listed in this ticket and had no luck.
4> Identify the root issue and solution:
4.a> run following from shell command (copy/pasted from error above) to reveal the root problem: the openssl was the problem in my case:
$ /var/folders/lj/vg60qx992j5frpmgbs498w940000gp/T/embedded-pg/PG-3d7ce5d05cd575a649dd635576931b19/bin/initdb
dyld: Library not loaded: /usr/local/opt/openssl/lib/libssl.1.0.0.dylib
Referenced from: /usr/local/Cellar/postgresql/11.4/lib/libpq.5.11.dylib
Reason: image not found
Abort trap: 6
4.b> Solution:
-- I have several versions of openssl installed, and a newer version openssl, 1.1.1d, is in use.
-- Solution: install or link to proper openssl version (in my case, 1.0.2s), which includes the proper libssl version being expected by postgresql (in my case, libssl.1.0.0.dylib).
4.c> Both command line and inteliJ work fine afterwards ;-)

5> The bottom line: some other programs may expect a newer/different version of openssl. May have to switch back and forth since i made the changes at system level.

@cwilbur12
Copy link

FYI for future people if you use mac and use brew they upgraded it automatically so force downgrading openssl worked in my case as well as @wtangv

@wmaciel
Copy link

wmaciel commented Feb 18, 2020

I'm having the same issue on Windows. I've verified that I have all of the mentioned MS VS C++ 2013 x64 and that running initdb manually works fine.

Not sure what else to try.

@pekevski
Copy link

pekevski commented Apr 2, 2020

running manually on my mac i get the following error:
dyld: Library not loaded: /usr/local/opt/openssl/lib/libssl.1.0.0.dylib Referenced from: /usr/local/lib/libpq.5.dylib Reason: image not found Abort trap: 6

@pekevski
Copy link

pekevski commented Apr 2, 2020

Running brew update && brew upgrade fixed the issue if you're a mac user with homebrew.

@anshulg5
Copy link

anshulg5 commented Apr 23, 2020

Setting the locale environment variables worked for me.
export LC_ALL="en_US.UTF-8"
export LC_CTYPE="en_US.UTF-8"

In macOS Catalina, set above variables in ~/etc/profile and restart your editor/open new shell.
https://stackoverflow.com/a/8161863/9878734

@apostolou
Copy link

Hello, I'm having this issue (Process [/tmp/embedded-pg/PG-6e6cb600c5af6f0b241f84df1594a9c6/bin/initdb, -A, trust, -U, postgres, -D, /tmp/epg924981102176549599, -E, UTF-8] failed
) when running my tests on bitbucket pipelines (I think it is using docker). I tried setting the locale environment variables but it didn't work.
Any ideas on how to fix it ?
Thx

@salluspeaks
Copy link

salluspeaks commented May 5, 2020

use a user that does not have root permission to execute embedded postgres. For some reason user with root access have problems during execution.
example to use in docker

##To change user to a non root user
RUN groupadd -r nonroot && useradd --no-log-init -r -m -g nonroot nonroot && chown -R nonroot:nonroot /src
# this will set the user to run the next commands
USER nonroot 

@apostolou
Copy link

thanks @salluspeaks for your reply! I'll test and let you know

@dubreuil534
Copy link

Had the same issue, even after I had installed Microsoft Visual C++ Redistributable for Visual Studio 2017 (64 bit).

What helped was to install MS Visual C++ Redistributable Packages for Visual Studio 2013 x64 :-)

Thanks!

Work for me thanks !!

@mdnajimahmed
Copy link

@dubreuil534 Thanks. MS Visual C++ Redistributable Packages for Visual Studio 2013 x64 worked for me too!

@shikhagoe
Copy link

shikhagoe commented Feb 14, 2021

Is there any specific version of redistributable c++ package..as for me nothing worked...changing encoding to utf8 of maven or cleaning temp or installing redistributable package

@rlapao
Copy link

rlapao commented Feb 17, 2021

I had a similar problem running tests with embedded postgres on MacOS M1:

java.lang.IllegalStateException: Process [/var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/embedded-pg/PG-4f38c8a8b2500189835f8ec1b75de81b/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/pg-embedded-009358be-3e95-4dbe-aa7a-01a9008511cb-3277843270072134584/epg524445573999525527, -E, UTF-8] failed

I copy pasted the command and run it on shell:

/var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/embedded-pg/PG-4f38c8a8b2500189835f8ec1b75de81b/bin/initdb -A trust -U postgres -D /var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/pg-embedded-009358be-3e95-4dbe-aa7a-01a9008511cb-3277843270072134584/epg524445573999525527 -E UTF-8

and the output show me a hint:

Running bootstrap script ... 2021-02-17 08:18:21.008 WET [28692] FATAL:  could not create shared memory segment: Cannot allocate memory
2021-02-17 08:18:21.008 WET [28692] DETAIL:  Failed system call was shmget(key=5432001, size=56, 03600).
2021-02-17 08:18:21.008 WET [28692] HINT:  This error usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMALL parameter.  You might need to reconfigure the kernel with larger SHMALL.
	The PostgreSQL documentation contains more information about shared memory configuration.
child process exited with exit code 1
initdb: removing contents of data directory "/var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/pg-embedded-cbe6f5f8-24b2-4431-afec-6e7aeb1dbb5d-8656684715588054403/epg6904898061245880853"

Then I run on shell:

sudo sysctl kern.sysv.shmmax=104857600
sudo sysctl kern.sysv.shmall=25600

And it worked.

@shikhagoe
Copy link

Thanks

I had a similar problem running tests with embedded postgres on MacOS M1:

java.lang.IllegalStateException: Process [/var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/embedded-pg/PG-4f38c8a8b2500189835f8ec1b75de81b/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/pg-embedded-009358be-3e95-4dbe-aa7a-01a9008511cb-3277843270072134584/epg524445573999525527, -E, UTF-8] failed

I copy pasted the command and run it on shell:

/var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/embedded-pg/PG-4f38c8a8b2500189835f8ec1b75de81b/bin/initdb -A trust -U postgres -D /var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/pg-embedded-009358be-3e95-4dbe-aa7a-01a9008511cb-3277843270072134584/epg524445573999525527 -E UTF-8

and the output show me a hint:

Running bootstrap script ... 2021-02-17 08:18:21.008 WET [28692] FATAL:  could not create shared memory segment: Cannot allocate memory
2021-02-17 08:18:21.008 WET [28692] DETAIL:  Failed system call was shmget(key=5432001, size=56, 03600).
2021-02-17 08:18:21.008 WET [28692] HINT:  This error usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMALL parameter.  You might need to reconfigure the kernel with larger SHMALL.
	The PostgreSQL documentation contains more information about shared memory configuration.
child process exited with exit code 1
initdb: removing contents of data directory "/var/folders/g_/jxzv9glj0p13njlzpr9xtcfm0000gn/T/pg-embedded-cbe6f5f8-24b2-4431-afec-6e7aeb1dbb5d-8656684715588054403/epg6904898061245880853"

Then I run on shell:

sudo sysctl kern.sysv.shmmax=104857600
sudo sysctl kern.sysv.shmall=25600

And it worked.

Thanks...but for me it's Windows.

@rc-ctream
Copy link

I faced this issue on mac and seems like it was a permission issue on intellij not being able to run commands. I was successfully able to run test case from terminal using gradle.

How do you fix it? I faced with the same propblem

@Crystark
Copy link

Crystark commented Nov 2, 2021

10:01:20.447 [DEBUG] [TestEventLogger] nyomio.commons.EmbeddedPostgresTest > testEmbedded() STANDARD_OUT
10:01:20.447 [DEBUG] [TestEventLogger]     10:01:20.442 [Test worker] INFO com.opentable.db.postgres.embedded.EmbeddedPostgres - Detected a Darwin x86_64 system
10:01:20.557 [DEBUG] [TestEventLogger]     10:01:20.557 [Test worker] INFO com.opentable.db.postgres.embedded.EmbeddedPostgres - Postgres binaries at /var/folders/4q/***/T/embedded-pg/PG-73dc0043fe7bdb624d5e8726bc457b7e
10:01:20.558 [DEBUG] [TestEventLogger]     10:01:20.558 [Test worker] DEBUG com.opentable.db.postgres.embedded.EmbeddedPostgres - 788f627d-e723-40a9-980a-f7b202965bb7 postgres: data directory is /var/folders/4q/***/T/epg7982076833040837593, postgres directory is /var/folders/4q/***/T/embedded-pg/PG-73dc0043fe7bdb624d5e8726bc457b7e
10:01:20.650 [DEBUG] [TestEventLogger]     10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - The files belonging to this database system will be owned by user "istvan".
10:01:20.650 [DEBUG] [TestEventLogger]     10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - This user must also own the server process.
10:01:20.650 [DEBUG] [TestEventLogger]     10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - 
10:01:20.650 [DEBUG] [TestEventLogger]     10:01:20.650 [log:pid(7092)] INFO init-788f627d-e723-40a9-980a-f7b202965bb7:initdb - initdb: invalid locale settings; check LANG and LC_* environment variables
10:01:20.671 [DEBUG] [TestEventLogger] 
10:01:20.671 [DEBUG] [TestEventLogger] nyomio.commons.EmbeddedPostgresTest > testEmbedded() FAILED
10:01:20.672 [DEBUG] [TestEventLogger]     java.lang.IllegalStateException: Process [/var/folders/4q/zq9fy2hs1kn3xcyt333x_l240000gn/T/embedded-pg/PG-***/bin/initdb, -A, trust, -U, postgres, -D, /var/folders/4q/***/T/epg7982076833040837593, -E, UTF-8] failed
10:01:20.672 [DEBUG] [TestEventLogger]         at com.opentable.db.postgres.embedded.EmbeddedPostgres.system(EmbeddedPostgres.java:602)
10:01:20.673 [DEBUG] [TestEventLogger]         at com.opentable.db.postgres.embedded.EmbeddedPostgres.initdb(EmbeddedPostgres.java:221)
10:01:20.673 [DEBUG] [TestEventLogger]         at com.opentable.db.postgres.embedded.EmbeddedPostgres.<init>(EmbeddedPostgres.java:142)
10:01:20.673 [DEBUG] [TestEventLogger]         at com.opentable.db.postgres.embedded.EmbeddedPostgres$Builder.start(EmbeddedPostgres.java:554)
10:01:20.673 [DEBUG] [TestEventLogger]         at com.opentable.db.postgres.embedded.EmbeddedPostgres.start(EmbeddedPostgres.java:438)
10:01:20.673 [DEBUG] [TestEventLogger]         at nyomio.commons.EmbeddedPostgresTest.testEmbedded(EmbeddedPostgresTest.kt:13)

@istvanszoboszlai Any chance you remember how you got such a neat output ? We're having issues debugging a gitlab CI pipeline and getting the output of the initdb command seems key to understanding what is missing.

@kumarks1122
Copy link

Updating brew fixed the issue brew update && brew upgrade

Thanks pekevski

@steve-todorov
Copy link

For people who are using PostgreSQL >= 15 under Windows 10+ you might need to install Visual C++ Redistributable for Visual Studio 2015. At least for me 2013 did not solve the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests